Bulk importing datasets

Upload a CSV or spreadsheet to register a dataset through the API.

Bulk importing datasets

You can register a dataset by uploading its file directly through the API. This is the same upload the Datasets page uses, exposed for scripts and data pipelines. The endpoint stores the file and creates a dataset record from the metadata you send with it.

This needs a plugin

Bulk upload is gated behind the dataset-bulk-upload plugin. If the plugin is not installed for your organization, the endpoint returns 403. An Admin can install it from the plugins marketplace.

The endpoint


Method and path	POST /api/dataset-bulk-upload/upload
Content type	multipart/form-data
Access	Admin or Editor, with the dataset-bulk-upload plugin installed

The request has two parts: the file itself in a form field named file, and an optional metadata field carrying a JSON object that describes the dataset.

Accepted files

The upload accepts CSV, XLS and XLSX files, up to 30 MB. The file is stored as-is. The endpoint does not parse rows or require particular column headers, so any valid CSV or spreadsheet is accepted; you organize the contents however your downstream process expects.

Making the request

Send the file in the file field. Put the dataset details in the metadata field as a JSON string.

bash

curl -X POST "http://localhost:3000/api/dataset-bulk-upload/upload" \
  -H "Authorization: Bearer <your-token>" \
  -F "file=@training-data.csv" \
  -F 'metadata={"name":"Training data Q2","type":"Training","classification":"Internal","contains_pii":false}'

Every metadata field is optional and has a default. The common ones:

Field	Description	Default
name	Display name for the dataset.	The file name
type	What the dataset is used for, for example Training.	Training
classification	Sensitivity classification.	Internal
contains_pii	Whether the dataset holds personal data (boolean).	false
description	Free-text description.	Empty
version	Dataset version string.	1.0
projects	Array of project ids to link the dataset to.	Empty
models	Array of model inventory ids to link the dataset to.	Empty

Response

A successful upload returns 201 with the new dataset id and the stored file id:

json

{
  "message": "Created",
  "data": {
    "datasetId": 42,
    "fileId": 108
  }
}

Errors

Code	When
400	No file was sent, or the metadata field was not valid JSON.
401	Missing or invalid token.
403	The dataset-bulk-upload plugin is not installed, or your role is not Admin or Editor.
413	The file is larger than 30 MB.
415	The file is not a CSV, XLS or XLSX.

Platform REST API

Auth, base URL, response shape and limits.

Datasets

Managing datasets in the app.

PreviousWorking with resources

NextAutomations API

Bulk importing datasets