Build on unilayer
unilayer is an open-source, self-hosted backend layer. It puts your data and files behind one secure API, with access of its own for every app, so your applications talk to a single, consistent interface instead of to databases and buckets directly.
Database tables you create and change at runtime through the API.
Versioned files behind a storage service, on local disk or S3, with on-the-fly image transforms.
An API user per app or service, with a role, policies and fine-grained permissions.
How it fits together
Your apps call unilayer over REST or GraphQL. unilayer stores structured data in PostgreSQL and caches hot reads and token lookups in Valkey. File content goes through the storage service, which runs next to the API and keeps the files on the local filesystem or any S3-compatible store such as MinIO. The API hands out signed transfer orders, and clients move the bytes with the storage service, or straight with S3.
Your apps (Blazor · Web · Mobile · Services) │ │ │ REST · GraphQL │ file content (signed URLs) ▼ ▼ unilayer API ── jobs ──▶ storage service │ │ ┌────┴─────┐ ▼ ▼ ▼ Local disk / S3 (MinIO) PostgreSQL Valkey
Quick start
This gets a single unilayer instance running locally with Docker Compose – PostgreSQL, Valkey, the API and the storage service – then creates your first collection.
Get the compose file
Take compose.yaml from the root of the repository. It uses the published images code.talaryon.dev/unilayer/unilayer and code.talaryon.dev/unilayer/unilayer-storage.
Create the signing key and the secrets
The API signs the transfer orders of the storage service with an EC P-256 key. Next to compose.yaml, create the key and a .env file. Pick a long random string for the static API token: it becomes the root user's token.
openssl genpkey -algorithm EC -pkeyopt ec_paramgen_curve:P-256 -out storage-signing-key.pem cat > .env <<EOF POSTGRES_PASSWORD=change-me UNILAYER_STATIC_TOKEN=my-root-token EOF
Start unilayer
docker compose up -d
The API listens on http://localhost:5380, the storage service on http://localhost:5390. The database schema is migrated automatically on startup. Check that everything is up:
curl http://localhost:5380/health
Create a collection and add data
TOKEN=my-root-token # Create a "tasks" collection – unilayer creates the table for you curl -X POST http://localhost:5380/api/v1/collections \ -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \ -d '{"name":"tasks","fields":[ {"name":"title","type":"TEXT","nullable":false}, {"name":"done","type":"BOOLEAN"}]}' # Add an item curl -X POST http://localhost:5380/api/v1/collections/tasks \ -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \ -d '{"title":"Try unilayer","done":false}' # Read it back curl -H "Authorization: Bearer $TOKEN" \ "http://localhost:5380/api/v1/collections/tasks?filter=done:eq:false"
http://localhost:5380/api/v1 for the interactive REST reference, or /api/v1/graphql/ui for the GraphQL explorer.
Requirements
| Component | Version | Purpose |
|---|---|---|
| .NET SDK / runtime | 10.0 | Build and run the server (not needed when using the Docker image) |
| PostgreSQL | 14+ | System tables and all collection data |
| Valkey | 7.0+ | Cache and token lookups (Redis-compatible); a node is not ready without it |
| Storage service | same release | Moves and transforms file content (unilayer/storage, image unilayer-storage) |
| S3 / MinIO | optional | Asset storage; required in cluster mode, at least for snapshots |
Installation
Backing services
Install PostgreSQL and Valkey with your platform's package manager, or run them as containers.
PostgreSQL
# macOS brew install postgresql@16 && brew services start postgresql@16 # Ubuntu / Debian sudo apt install postgresql postgresql-contrib sudo systemctl enable --now postgresql
Then create a database and user:
CREATE DATABASE unilayer;
CREATE USER unilayer_user WITH PASSWORD 'your_secure_password';
GRANT ALL PRIVILEGES ON DATABASE unilayer TO unilayer_user;
Valkey
# macOS brew install valkey && brew services start valkey # Ubuntu / Debian sudo apt install valkey-server sudo systemctl enable --now valkey-server # Docker docker run -d --name valkey -p 6379:6379 -v valkey-data:/data valkey/valkey
MinIO (optional)
docker run -d -p 9000:9000 -p 9001:9001 minio/minio server /data --console-address ":9001"
Run from source
The .NET client, the shared models and the storage service are submodules (libs/client, libs/storage), so clone with them:
git clone --recurse-submodules https://code.talaryon.dev/unilayer/unilayer
# or, in an existing clone
git submodule update --init
Create an appsettings.yaml (see Configuration) in the working directory, then:
dotnet run --project src/unilayer
The API listens on http://localhost:5380 by default. Start the storage service next to it and point it at the API's public keys:
dotnet run --project libs/storage/src/unilayer.Storage.Host -- \ --storage:keys:url=http://localhost:5380/api/v1/storage/keys
Run with Docker
The published images are code.talaryon.dev/unilayer/unilayer and code.talaryon.dev/unilayer/unilayer-storage; both listen on port 8080. compose.yaml shows how they fit together. To build the API image yourself:
dotnet publish src/unilayer -c Release -o publish docker build -t unilayer .
The image ships with pg_dump/pg_restore for cluster snapshots. They must be at least as new as your PostgreSQL server; the default is version 18. Override it with --build-arg PG_CLIENT_VERSION=<major>.
Configuration
unilayer reads its settings from three places:
- YAML file:
appsettings.yamlin the working directory is loaded automatically. Point to another file with--config=<path>. - Environment variables: every key has an
UNILAYER_*equivalent, which suits containers. - Command-line arguments: e.g.
--valkey:host=cache.local.
Need a YAML file from environment variables? scripts/generate-config.sh <output.yaml> builds one for you.
Example appsettings.yaml
endpoint: "0.0.0.0:5380" authentication: staticApiToken: "your-secure-api-token" database: providerType: "PostgreSQL" connectionString: "Host=localhost;Database=unilayer;Username=unilayer_user;Password=...;Port=5432" valkey: host: "localhost" port: 6379 password: "" database: 0 storage: provider: "service" # or "local", "minio" / "s3", "none" service: url: "http://localhost:5390" signing: privateKeyFile: "./storage-signing-key.pem" # s3: # endpoint: "http://localhost:9000" # accessKey: "minioadmin" # secretKey: "minioadmin" # bucketName: "unilayer-assets"
Core
| Key | Environment variable | Default | Description |
|---|---|---|---|
endpoint | UNILAYER_ENDPOINT | 0.0.0.0:5380 | Address and port the API listens on |
authentication:staticApiToken | UNILAYER_AUTHENTICATION_STATICAPITOKEN | – | Admin token, and the root user's token (see permissions). Plaintext or hashed |
authentication:staticApiTokenSecondary | UNILAYER_AUTHENTICATION_STATICAPITOKENSECONDARY | – | A second static token accepted next to the first, for rotating it |
tls:certificatePath | UNILAYER_TLS_CERTIFICATEPATH | – | PKCS#12 certificate. When set, the API is served over HTTPS |
tls:certificatePassword | UNILAYER_TLS_CERTIFICATEPASSWORD | – | Password of the certificate |
tls:reloadIntervalSeconds | UNILAYER_TLS_RELOADINTERVALSECONDS | 30 | How often the certificate files are checked for a renewal. 0: never, a renewal needs a restart |
rateLimit:globalPerMinute | UNILAYER_RATELIMIT_GLOBALPERMINUTE | 1000 | Requests per client and minute (see Security) |
rateLimit:writesPerMinute | UNILAYER_RATELIMIT_WRITESPERMINUTE | 50 | Writes per client and minute |
idempotency:enabled | UNILAYER_IDEMPOTENCY_ENABLED | true | Accept the Idempotency-Key header (see Security) |
idempotency:retentionHours | UNILAYER_IDEMPOTENCY_RETENTIONHOURS | 24 | How long keys are remembered |
idempotency:maxResponseBytes | UNILAYER_IDEMPOTENCY_MAXRESPONSEBYTES | 64 KB | Larger responses are replayed with their status only |
compression:enabled | UNILAYER_COMPRESSION_ENABLED | true | Brotli/gzip for JSON and other text responses. Turn it off when a reverse proxy compresses |
compression:level | UNILAYER_COMPRESSION_LEVEL | fastest | fastest, optimal or smallest |
compression:mimeTypes | UNILAYER_COMPRESSION_MIMETYPES | text-like types | Comma-separated MIME types that replace the default list |
metrics:endpoint | UNILAYER_METRICS_ENDPOINT | – | A separate listener for /metrics, e.g. 0.0.0.0:9464 (see Operations) |
Database
| Key | Environment variable | Default | Description |
|---|---|---|---|
database:providerType | UNILAYER_DATABASE_PROVIDERTYPE | PostgreSQL | Database provider |
database:connectionString | UNILAYER_DATABASE_CONNECTIONSTRING | – | Npgsql connection string |
Valkey
| Key | Environment variable | Default |
|---|---|---|
valkey:host | UNILAYER_VALKEY_HOST | localhost |
valkey:port | UNILAYER_VALKEY_PORT | 6379 |
valkey:password | UNILAYER_VALKEY_PASSWORD | – |
valkey:database | UNILAYER_VALKEY_DATABASE | 0 |
valkey:ssl | UNILAYER_VALKEY_SSL | false |
valkey:connectTimeout | UNILAYER_VALKEY_CONNECTTIMEOUT | 5000 ms |
valkey:syncTimeout | UNILAYER_VALKEY_SYNCTIMEOUT | 5000 ms |
Storage
| Key | Environment variable | Default | Description |
|---|---|---|---|
storage:provider | UNILAYER_STORAGE_PROVIDER | local | service (only the storage service reaches the files), local, minio / s3 (any S3-compatible store) or none (no assets) |
storage:quotaBytes | UNILAYER_STORAGE_QUOTABYTES | 0 | The most all assets may take together; 0 for no limit. Beyond it uploads get 507 |
local:path | UNILAYER_LOCAL_PATH | ./storage/assets | Base directory for local storage |
s3:endpoint | UNILAYER_S3_ENDPOINT | – | S3 / MinIO endpoint URL |
s3:accessKey | UNILAYER_S3_ACCESSKEY | – | Access key |
s3:secretKey | UNILAYER_S3_SECRETKEY | – | Secret key |
s3:bucketName | UNILAYER_S3_BUCKETNAME | unilayer-assets | Bucket name |
s3:region | UNILAYER_S3_REGION | us-east-1 | Region |
s3:basePath | UNILAYER_S3_BASEPATH | assets | Key prefix inside the bucket |
s3:useSsl | UNILAYER_S3_USESSL | false | Use HTTPS towards the store |
s3:createBucketIfNotExists | UNILAYER_S3_CREATEBUCKETIFNOTEXISTS | true | Create the bucket on startup |
s3:publicUrlBase | UNILAYER_S3_PUBLICURLBASE | – | Public base URL for asset links |
The storage service reads the same local:path and s3:* keys, so both reach the same files. The settings of the storage service, transfers and signing keys are in Uploads & downloads; file types, image transforms, optimization and retention in Assets; GraphQL and cluster settings in GraphQL and Cluster mode.
Collections
A collection is a real PostgreSQL table that you define through the API. Creating, changing or deleting a collection changes the table at runtime; there are no migrations to write. Every collection has explicitly typed fields.
Field types
| Type | Aliases | Holds |
|---|---|---|
TEXT | STRING, VARCHAR, CHAR, NCHAR, NVARCHAR | Text. Returned exactly as stored, even if it looks like JSON |
INTEGER | INT, SMALLINT, BIGINT, LONG | Whole numbers (stored as BIGINT) |
REAL | FLOAT, DOUBLE, DECIMAL, NUMERIC, NUMBER | Exact decimal numbers (stored as NUMERIC) |
BOOLEAN | BIT | True / false |
DATETIME | TIMESTAMP | A point in time, stored with time zone and returned in UTC |
DATE | – | A calendar date |
TIME | – | A time of day |
UUID | – | A UUID; the type of every reference |
BYTES | BYTEA, BLOB | Binary data |
JSON | JSONB | A JSON object or array, returned as JSON |
Fields can also be nullable or required, have a default value, be unique, and be indexed (isIndexed) for fast lookups and filters. Field names are checked against PostgreSQL reserved words.
System fields
unilayer manages four fields on every collection. The unilayer_ prefix is reserved, so you can't define fields that start with it.
| Field | Type | Meaning |
|---|---|---|
unilayer_id | UUID | Primary key, a time-ordered UUIDv7 |
unilayer_index | BIGSERIAL | Insertion order, the default sort |
unilayer_created_at | TIMESTAMP | Creation time, in UTC |
unilayer_updated_at | TIMESTAMP | Last update in UTC, or null |
Responses keep these separate from your data: your fields are in data and system values are in metadata, so names can never collide.
Relationships
Mark a UUID field with isReference: true and a referenceCollection to create a foreign key. A reference always points at the target's unilayer_id; with cascadeDelete: true, deleting the target deletes the referencing items too. When you list items, referenced records are expanded and nested automatically, up to 3 levels deep. References into collections the caller can't read stay plain ids, and reading a single item by its id doesn't expand them.
{
"name": "users",
"fields": [
{ "name": "username", "type": "TEXT" },
{ "name": "profileImage", "type": "UUID",
"isReference": true, "referenceCollection": "images" }
]
}
Bulk writes and counts
Up to 1,000 items can be added, updated or deleted in one request, in one transaction: POST /collections/{name}/bulk, PUT /collections/{name}/bulk and POST /collections/{name}/bulk/delete. GET /collections/{name}/count?filter=… counts the matching items. See the API reference.
Schema history
Every schema change is written to a system history. You can export this history and replay it on another installation to move a collection structure between sites. See Operations.
Querying
The same query language works on every collection: filter, sort, paginate and pick fields using query parameters.
| Parameter | Example | Meaning |
|---|---|---|
filter | age:gt:18,status:eq:active | Conditions: , means AND, | means OR |
sort | name,unilayer_created_at:desc | One or more fields, including the system fields, :asc / :desc |
limit | 20 | Page size, 1–1000 (default 100) |
offset | 40 | Items to skip |
fields | name,profileImage.url | Field selection. A dotted path expands a reference; system values are always in metadata |
Filter operators
Filters use the form field:operator:value.
| Operator | Meaning | Example |
|---|---|---|
eq / ne | Equals / not equals | status:ne:deleted |
gt / ge / gte | Greater than (or equal) | price:gte:100 |
lt / le / lte | Less than (or equal) | age:lt:65 |
like | Pattern match | name:like:%John% |
in | Value in list | status:in:active\,pending |
isnull / notnull | Null checks | deletedAt:isnull: |
A backslash escapes ,, | and : in values, so the values of in are separated by \, – a plain comma starts the next condition. like compares the field as text. Points in time are compared as instants; a value without an offset is UTC.
Values are parsed according to the field's type and always sent to the database as parameters; a value that doesn't parse gives 400. Field names and operators are checked against the schema and an allow-list, so filters are safe from SQL injection.
Assets
Assets are files with metadata. Each asset has a UID, name, file type, MIME type, size, a SHA-256 checksum and a version number that increases every time the content is replaced. You can attach custom metadata as JSON. Deleting an asset is a soft delete, so it can be restored (assets:write) until it is purged.
File types
An asset's type comes from its file extension:
| Type | Extensions |
|---|---|
Document | pdf, docx, pptx |
Image | png, jpg, jpeg, gif, svg, webp, tif, tiff, heic |
Video | mp4, mov |
Audio | m4a, mp3, ogg, opus, wav |
Text | txt or no extension |
Spreadsheet | xlsx, csv |
Archive | zip, stored as it is |
Other | Extensions allowed through assets:allowedExtensions that fit no other type |
Other extensions are refused; executables and scripts always are. The first bytes of an upload must fit its extension (a .png must start like a PNG), otherwise it is refused with 400. The default upload limit through the API is 100 MB per file.
| Key | Default | Description |
|---|---|---|
assets:allowedExtensions | – | Extensions allowed in addition, e.g. dwg,step |
assets:verifyContent | true | Whether uploads must match their extension |
Image transforms
Images can be resized and re-encoded on download by adding query parameters to the asset URL, so there is no need to store separate thumbnails.
/api/v1/assets/{uid}?width=200&format=webp
/api/v1/assets/{uid}?width=300&height=300&fit=cover&quality=70
/api/v1/assets/{uid}?scale=0.3
| Parameter | Values |
|---|---|
width, height | Target size in px. Aspect ratio is kept when only one is set |
scale | Multiplier, e.g. 0.3. Can't be combined with width/height |
fit | contain (default), cover, stretch |
format | webp, png, jpg, gif |
quality | 1–100 (default 80), for jpg and webp |
- EXIF, XMP, IPTC and ICC data (such as GPS positions) is stripped from transformed images.
- Variants up to 1 MiB are cached per asset version, so uploading new content invalidates them automatically. Responses support
ETag/304. - Outputs are limited to 4096 px per side and sources to 50 megapixels. SVGs are returned unchanged.
- Transforms run in a limited number of slots per node; a request that waits too long gets
503withRetry-After. The limits are settings underassets:transform:*.
Optimized images
With assets:optimize:enabled, JPEG, PNG and GIF assets get a lossless optimized copy in the background: the same pixels and metadata, typically 5–30 % smaller. Downloads without modifiers serve the copy; ?original=true always returns the stored file, which is what Size and Checksum describe. Every copy is verified against its original before it is kept. Run the server once with --remove-optimized-assets to remove the copies again.
Compression
Text-like downloads (text, CSV, JSON, XML, SVG, …) are compressed with Brotli or gzip when the client accepts it. Images, video, archives and office documents are sent as stored.
Purge and retention
DELETE /api/v1/assets/{uid}?purge=true removes a deleted asset for good: its content, copies, variants and record. It needs assets:purge, and the asset must have been deleted long enough ago that no kept backup still holds its record.
| Key | Default | Description |
|---|---|---|
assets:retention:minAgeDays | 30 | How long an asset stays deleted before it may be purged |
assets:retention:purgeBefore | – | Only assets deleted before this time may be purged, e.g. your oldest kept backup |
assets:retention:purgeAfterDays | 0 | Purge deleted assets automatically after this many days; 0 turns the job off |
assets:retention:intervalHours | 24 | How often the retention job runs |
Uploads & downloads
With the storage service running next to the API, file content no longer passes through the API and isn't base64-encoded. The API hands out a transfer order – a URL with a signed token – and the client moves the bytes with the storage service. The .NET client does all of this for you (UploadAsync, ResumeUploadAsync, GetDownloadUrlAsync).
Upload
- Create the asset with
POST /api/v1/assets. POST /api/v1/assets/{uid}/upload-sessionwith the file'ssizeandsha256. The type, size and quota are checked here. The answer has asessionand atransferorder (URL, method, headers, expiry).- Send the file to the order's URL. From 16 MiB the order comes in parts: upload them in any order and in parallel, then post the list of parts with their ETags.
POST /api/v1/assets/{uid}/upload-session/completewith the session. The API checks the file and returns the metadata with the new version.
An order works for 15 minutes. …/upload-session/renew hands out a new one for the same upload, keeping the parts already stored, so long and interrupted uploads can be resumed. Uploads that are never completed are cleaned up after a day.
Download
GET /api/v1/assets/{uid}/download-url returns a GET order for the content, the optimized copy (unless original=true) or, with image modifiers, a variant. The URL works without a token until it expires and is stable within a time window, so browsers and CDNs can cache it. The storage service answers Range requests.
Quotas
Besides the global storage:quotaBytes, an upload session may carry a quotaScope (e.g. a workspace id) and your application's quotaLimitBytes for it. An upload that would take the scope over its limit is refused with 507. Open uploads count with their announced size, so parallel uploads can't overbook. GET /api/v1/assets/quota?scope=… tells what a scope takes.
Straight to S3
With storage:transfer:mode=direct and an S3 store your clients can reach, the API hands out presigned S3 URLs instead, and content goes between the client and S3 directly. The orders look the same, so clients don't change. The bucket needs CORS for browser clients and a lifecycle rule that aborts incomplete multipart uploads.
Settings of the API
| Key | Default | Description |
|---|---|---|
storage:service:url | – | The storage service as clients reach it; turns the endpoints on (otherwise they answer 501) |
storage:service:internalUrl | storage:service:url | Where the API itself reaches the service, e.g. http://storage:8080 |
storage:signing:privateKey / privateKeyFile | – | The ES256 key (EC P-256, PEM) orders are signed with; the same on every node |
storage:signing:keyId | unilayer-1 | Change it together with the key |
storage:signing:standbyPrivateKey, …File, standbyKeyId | – | A second key during a rotation: published and accepted, never signed with |
storage:signing:issuer | unilayer | The name the storage service knows this instance by |
storage:transfer:mode | gateway | direct: straight to S3 with presigned URLs |
storage:transfer:directEndpoint | s3:endpoint | The S3 store as clients reach it |
storage:transfer:apiUrl | the request's | The API as clients reach it, behind a proxy or in a cluster |
storage:transfer:uploadExpiryMinutes | 15 | How long an upload order works |
storage:transfer:downloadWindowMinutes | 60 | Window in which a download URL stays the same |
storage:multipart:thresholdBytes / partSizeBytes | 16 MiB | From which size uploads go in parts, and how large a part is |
storage:legacy:sunset | – | Date sent as the Sunset header of the deprecated content endpoints |
Environment variables follow the usual pattern, e.g. UNILAYER_STORAGE_SERVICE_URL. To rotate the signing key without downtime: set the new key as standby, swap the two (key ids included), and remove the standby key after a day.
Settings of the storage service
| Key | Default | Description |
|---|---|---|
storage:provider | local | local (one node only) or s3 / minio, with the same local:path or s3:* as the API |
storage:keys:url | – | The API's public keys, GET /api/v1/storage/keys |
storage:cors:origins | none | Web origins allowed to call it; * for any |
storage:valkey:configuration | none | A Valkey for the job queue and variant cache, e.g. the instance's |
storage:maxUploadBytes | 10 GiB | The largest upload, whatever a token allows |
storage:mode | single | cluster for several storage nodes behind a load balancer (S3 and a Valkey of its own required) |
PUT /api/v1/assets/{uid} with base64 content, and the content of GET /api/v1/assets/{uid} – still work, but are deprecated once the storage service is on. They answer with Deprecation and Link headers pointing at their successors.Users & permissions
Every request is authenticated with a bearer token:
Authorization: Bearer <token>
The model
A named list of permission strings.
Bundles one or more policies.
Has exactly one role and its own API token.
Permission strings
Permissions follow the form resource[:scope[:name]]:action. A write permission includes read.
| Permission | Grants |
|---|---|
collections:read / :write | All collections |
collections:{scope}:read | Collections with a scope prefix |
collections:{scope}:{name}:write | One specific collection |
assets:read / :write | Assets |
assets:purge | Remove deleted assets for good |
users:read / :write | User management |
security:read / :write | Roles and policies |
cluster:admin | Cluster membership and status |
Nobody can grant more than they hold: creating or changing a policy, assigning a policy to a role, or giving a user a role requires the caller to hold every permission involved.
First start
On first startup unilayer creates three immutable entities:
fullaccesspolicy, which contains every permission. Permissions added by later versions are added to it at the next start.adminrole, which is assigned thefullaccesspolicy.rootuser, which has theadminrole and usesauthentication:staticApiTokenas its token.
authentication:staticApiToken is set, it authenticates as an admin, and the root user is keyed to it at every start. Only its hash is stored in the database. Use it to create real users, then keep it somewhere safe. To replace it without downtime, set the new token as authentication:staticApiTokenSecondary, switch your clients, then make it the primary token.GraphQL
unilayer serves a GraphQL API next to REST. The schema is generated from your collections and rebuilt automatically when a collection or field changes. Permissions and write paths are the same as for REST.
- Endpoint:
POST /api/v1/graphql. Authentication is required, including for introspection. - Explorer:
/api/v1/graphql/ui, served when bothenableUiandenableIntrospectionare on. Add yourAuthorizationheader in its headers panel. - SDL download:
GET /api/v1/graphql/schema, for codegen.
query { postsList(where: [{ field: views, op: GTE, value: "5" }], orderBy: [{ field: title }], limit: 10) { items { id title views createdAt } hasMore } } mutation { createPosts(input: { title: "Hello" }) { id } }
| Key | Default | Description |
|---|---|---|
graphql:enabled | true | Turn GraphQL on or off |
graphql:path | /api/v1/graphql | Endpoint path |
graphql:enableUi | true | Serve the explorer |
graphql:enableIntrospection | true | Allow schema introspection |
graphql:maxQueryDepth | 10 | Maximum query depth |
graphql:maxQueryComplexity | 5000 | Maximum query complexity |
graphql:defaultPageSize / maxPageSize | 20 / 100 | Paging |
graphql:collectionNamePatterns | * | Collections to expose (wildcards) |
graphql:excludedCollections | system_* | Collections to hide (wildcards) |
Environment variables follow the usual pattern, e.g. UNILAYER_GRAPHQL_ENABLED. Mutations count against the same write rate limit as REST.
.NET client
Unilayer.Client is a typed client for .NET 10, MIT-licensed and developed in its own repository. It maps your classes to collections with attributes and turns LINQ into server-side queries. The wire-format models ship as Unilayer.Models, released with the same version.
1. Install and register
dotnet add package Unilayer.Client
builder.Services.AddUnilayerClient(options =>
{
options.Url = "https://unilayer.example.com";
options.Token = builder.Configuration["Unilayer:Token"];
});
// Several backends? Register named clients and use IUnilayerClientFactory.
builder.Services.AddUnilayerClient("Archive", options => { /* ... */ });
2. Describe your model
[UniCollection("products")] public class Product { [UniField("name", IsRequired = true)] public string Name { get; set; } = ""; [UniField("sku", IsUnique = true)] public string Sku { get; set; } = ""; [UniField("price", IsIndexed = true)] public decimal Price { get; set; } }
3. Sync and query
await client.SyncCollectionSchemaAsync<Product>(); // creates / updates the collection await client.CreateAsync(new Product { Name = "Widget", Sku = "W-1", Price = 9.90m }); var cheap = await client.Query<Product>() .Where(p => p.Price < 20) .OrderBy(p => p.Name) .Take(50) .ToListAsync();
Bulk writes and files
BulkCreateAsync, BulkUpdateAsync and BulkDeleteAsync write up to 1,000 items in one transaction. UploadAsync uploads through the storage service, in parallel parts for large files, and ResumeUploadAsync continues an interrupted upload. GetDownloadUrlAsync returns a download URL, optionally for an image variant or the original. Responses are compressed.
Field encryption
Mark a property with IsEncrypted = true and set FieldEncryptionMasterKeyBase64 (32 random bytes, base64-encoded). Values are encrypted with AES-256-GCM in the client, so the server only ever stores ciphertext.
Built-in resilience
The client retries with exponential backoff (3 retries by default), has a circuit breaker and uses a 30 s timeout. All of these can be configured on UnilayerClientOptions. It also supports schema versioning and migrations for evolving models.
Security
- HTTPS: set
tls:certificatePathand the API listener serves HTTPS only. In cluster mode this is mandatory. Renewed certificate files are picked up without a restart. - Tokens: only hashes of API tokens are stored. Newly issued tokens are never kept for idempotent replays.
- Rate limiting per client: 1000 requests/min globally and 50/min for writes (every write endpoint and GraphQL mutations), both configurable. A request over the limit gets
429withRetry-Afterat once. Failed token attempts are throttled separately, at 30 per client and minute. - Safe retries: send an
Idempotency-Keyheader with a write, and a retry after a timeout or a503gets the first response again (Idempotent-Replay: true) instead of being applied twice. - Least privilege: give each app its own user and a role that is scoped down to the collections it needs.
- Sensitive data: use client-side field encryption for values the server should never see in plain text.
Cluster mode
For high availability, run several unilayer nodes. Each node has its own PostgreSQL, Valkey and WAL directory, and all nodes share one S3 store. The nodes elect a leader with Raft, and writes are replicated through the Raft log to every node. A write sent to a follower is forwarded to the leader, and reads are served locally.
Requirements
A node refuses to start unless all of these are true:
storage:providerisservice,minioors3, ands3:*is set: snapshots (and, without the storage service, the assets) live in the shared store.- The public API has a TLS certificate (
tls:certificatePath). - Node-to-node traffic uses mutual TLS with a private CA.
Certificates
CERT_PASSWORD=secret ./scripts/generate-cluster-certs.sh ./certs node1 node2 node3
Give each node its own .pfx and ca.crt, and keep ca.key off the nodes. Certificate files are reloaded in place, so they can be renewed without a restart. When a certificate has less than 30 days left, /health reports Degraded so you can alert on it in time.
Settings
| Key | Default | Description |
|---|---|---|
cluster:enabled | false | Turn cluster mode on |
cluster:nodeId | – | Unique node name |
cluster:bindEndpoint | – | This node's address. Must be reachable (not 0.0.0.0). A node that isn't in peers joins later |
cluster:peers | – | The members a node starts with the first time, as a YAML list or comma-separated. Use either all IPs or all hostnames |
cluster:walDirectory | – | Raft write-ahead log directory |
cluster:internalApiPort | – | Internal mTLS-only port, the same on every node |
cluster:tlsCertificatePath | – | Node .pfx (serverAuth + clientAuth) |
cluster:tlsCertificatePassword | – | Password of the .pfx |
cluster:tlsCaCertificatePath | – | CA public certificate |
cluster:valkeyLocalEndpoint | – | This node's Valkey, which switches to primary or replica with leadership |
cluster:lowerElectionTimeoutMs / upper… | 500 / 1000 | Election timeouts |
cluster:connectTimeoutMs | half of lower | Must be shorter than the lower election timeout |
cluster:snapshotIntervalMinutes | 1440 | pg_dump snapshots to S3. 0 disables them |
cluster:snapshotPrefix | cluster-snapshots/ | Key prefix of the snapshots |
cluster:initialSnapshot | true | A leader without a snapshot takes one right away |
cluster:requestTimeoutMs | 60000 | How long a forwarded write may take |
cluster:readyMaxLag | 100 | A node further behind than this many entries is not ready |
cluster:valkeyPort | 6379 | Port of the nodes' Valkeys |
Environment variables follow the usual pattern, e.g. UNILAYER_CLUSTER_PEERS=10.0.0.1:7000,10.0.0.2:7000,10.0.0.3:7000.
Members
cluster:peers is only the list a node starts with the first time. From then on the member list is part of the replicated log and changes only through the membership API, which needs cluster:admin:
| Call | Does |
|---|---|
GET /api/v1/cluster/status | Role, leader, term, log indexes, snapshot and members |
GET /api/v1/cluster/members | The members |
POST /api/v1/cluster/members | Adds a node ({"endpoint":"host:port"}); it is brought up to date from a snapshot |
DELETE /api/v1/cluster/members/{host:port} | Removes a node |
POST /api/v1/cluster/resign | The leader gives up leadership, e.g. before restarting it |
To add a node, start it with cluster:peers set to the current members, without itself, then add it through the API. A node that lost its disks just starts again under the same address and is restored from the latest snapshot.
Storage service
Every node uses the storage service the same way, with the same signing key, and keeps s3:* for its snapshots. Only the leader sends jobs to the service. For high availability, run the storage service as a cluster of its own: storage:mode=cluster, S3 and a Valkey of its own on every storage node.
Consistency
- A write is acknowledged once a majority has it and the leader has applied it.
- Writes are checked before they are replicated, and a write that fails when the leader applies it fails the request, as it would on a single node.
- Writes are fenced by leadership. A non-leader rejects writes with
503. After a503caused by lost leadership, the write's outcome is unknown: send writes with anIdempotency-Keyand retry with the same key. The key travels through the log, so the write is never applied twice. - Reads on a follower can lag by about one heartbeat (~250 ms by default). To read your own write, read from the leader.
masterauth to that password on every instance so replicas can follow the current primary.Operations
Health checks
| Path | Use |
|---|---|
/health | Overall status with all checks: database, Valkey, storage or storage service, and cluster |
/health/live | Liveness probe (database) |
/health/ready | Readiness probe: database, Valkey, storage service and, in cluster mode, whether the node has joined, knows a leader and has caught up |
Metrics
With metrics:endpoint (e.g. 0.0.0.0:9464) the server opens a second listener that serves /metrics in the Prometheus format: requests and their duration, database and asset sizes, image optimization, and in cluster mode role, leader, log indexes, replication lag, snapshots and certificate expiry. It is plain HTTP without authentication, so keep it inside your monitoring network.
API reference
Every instance serves an interactive REST reference at /api/v1, and the OpenAPI document at /api/v1/openapi/{document}.json.
Moving schemas between sites
Export the collection schema history from one installation and replay it against another installation's API:
# On the source (same configuration as the running instance) dotnet unilayer.dll --export-history-file=schema.jsonl # Replay against the target dotnet unilayer.dll --import-history-file=schema.jsonl \ --import-target-url=https://target.example.com \ --import-target-token=$TARGET_TOKEN
Use --export-from-sequence=<n> and --import-from-sequence=<n> to do this incrementally or to resume a replay.
Maintenance commands
--truncate-all drops the tables of all collections with their metadata and cache – users, permissions and history stay. --reinitialize-database drops every table. Both destroy data, refuse to run in cluster mode, run once and then exit. Use them only on development instances or with a backup.--remove-optimized-assets removes the optimized copies of images, e.g. after turning optimization off.