Datasets Upload: CSV and JSON

Veilio datasets support both CSV and JSON upload formats.

Veilio datasets let you upload structured files, tokenize sensitive columns, and share filtered views with role-based access policies.

Supported formats

• CSV (.csv)
• JSON (.json) — array of objects

Expected JSON shape

[
  { "email": "[e-mail]", "phone": "[phone]", "customer_id": "C001" },
  { "email": "[e-mail]", "phone": "[phone]", "customer_id": "C002" }
]

---

Upload flow

1. Owner or DPO uploads a file from the dashboard or via API.
2. Veilio extracts headers / columns from CSV or JSON.
3. columnPolicies must include every column exactly once.
4. Non-empty cell values are tokenized into the dataset grid.
5. A dataset version is created and available for controlled download.

Identity column (entityId)

Specify which column holds your business identifier (kept in clear text, links tokens to a profile):

• Dashboard: set during upload configuration.
• API: entityIdColumn in the multipart form (must match a column name).

Historical migration upload

For large legacy imports, pass migration=true so tokenized cells consume the migration budget instead of monthly plan quota.

Requirements:

• Migration mode activated (POST /api/onboarding/migration).
• Active Starter / Pro / Enterprise plan (or MIGRATION API key).

---

API upload (multipart)

Endpoint: POST /api/datasets

Auth: API key (Authorization: Bearer or X-API-Key) or dashboard session.

Role: Owner or DPO only.

Content-Type: multipart/form-data

Form field	Required	Description
file	Yes	CSV or JSON file
name	Yes	Dataset name (1–200 chars)
columnPolicies	Yes	JSON string — one policy per column
format	No	csv or json (auto-detected from filename if omitted)
description	No	Max 2000 chars
entityIdColumn	No	Column name for business ID (plaintext)
migration	No	"true" to use migration budget

Example columnPolicies

{
  "customer_id": { "roles": ["OWNER", "DPO", "MEMBER", "VIEWER"] },
  "email": { "roles": ["OWNER", "DPO"] },
  "phone": { "roles": ["OWNER", "DPO", "MEMBER"] }
}

• roles: which organization roles may see this column on download.
• userIds: optional list of specific user IDs (alternative to roles).

Every column in the file must have a policy entry. Extra or missing keys return VALIDATION_ERROR.

curl example

export VEILIO_API_KEY="veil_live_xxxxxxxx"
export VEILIO_BASE_URL="https://app.veilio.xyz/api"

curl -s -X POST "$VEILIO_BASE_URL/datasets" \
  -H "Authorization: Bearer $VEILIO_API_KEY" \
  -F "file=@contacts.csv" \
  -F "name=CRM Export March 2026" \
  -F 'description=Historical customer export' \
  -F 'entityIdColumn=customer_id' \
  -F 'migration=true' \
  -F 'columnPolicies={
    "customer_id": { "roles": ["OWNER", "DPO", "MEMBER", "VIEWER"] },
    "email": { "roles": ["OWNER", "DPO"] },
    "phone": { "roles": ["OWNER", "DPO", "MEMBER"] }
  }'

JSON file example

curl -s -X POST "$VEILIO_BASE_URL/datasets" \
  -H "Authorization: Bearer $VEILIO_API_KEY" \
  -F "file=@contacts.json;type=application/json" \
  -F "name=Contacts JSON" \
  -F 'format=json' \
  -F 'columnPolicies={"email":{"roles":["OWNER","DPO"]},"phone":{"roles":["OWNER","DPO","MEMBER"]}}'

Success response 201

{
  "id": "ds_...",
  "name": "CRM Export March 2026",
  "version": 1,
  "byteSize": 1048576,
  "columnCount": 3,
  "storage": "tokenized",
  "shareApiKey": "veil_...",
  "shareApiKeyPrefix": "veil_ab",
  "datasetAccessToken": "dsat_...",
  "datasetAccessTokenExpiresAt": "2026-03-28T10:00:00.000Z",
  "datasetAccessTokenHeader": "x-veilio-dataset-access-token"
}

shareApiKey and datasetAccessToken are returned only for organization datasets.

---

Validation rules and limits

Rule	Default / detail
Max file size	320 MB (VEILIO_DATASET_MAX_BYTES)
Max cells	200 000 (headers × rows, VEILIO_DATASET_MAX_CELLS)
Empty file	Rejected
columnPolicies	Must match columns exactly
JSON shape	Non-empty array of objects

Common errors

Message	Cause
JSON dataset must be an array of objects	Invalid JSON structure
JSON dataset is empty	No rows
Missing policy for column "..."	Incomplete columnPolicies
Policy references unknown column "..."	Extra policy key
Too many cells (max 200000)	File too large in cell count
File exceeds max size (... bytes)	File too large in bytes
Only Owner or DPO can upload datasets	Insufficient role
PLAN_LIMIT	Dataset or token quota exceeded
Migration budget exhausted / expired	When migration=true

---

Download behavior

Downloaded datasets are filtered by role and column policy. Consumers only receive columns they are allowed to see.

For programmatic download, use a Dataset Access Token (dsat_...) in header x-veilio-dataset-access-token.

Related links

• Dataset Access Tokens
• Historical Migration Mode
• API Reference