Object Store¶
Move files in and out of S3 or any S3-compatible store: put, get, list, or hand out a temporary link.
What it is¶
Files, in and out of an object store: put one in, read one back, list what is there, or hand somebody a temporary link. It speaks the S3 REST API, so the same step works against Amazon S3, MinIO, Cloudflare R2, Backblaze B2 or anything else S3-compatible. Which one it is comes from the connection, not from the step.
How it works¶
Pick a connection (it carries the keys, the region, the bucket and, for a non-Amazon store, the endpoint), choose an operation and give a key. The key is a path inside the bucket, and references resolve, so name the actual thing: returns/{{order.data.id}}/photo.jpg.
Put stores content you write inline, or copies whatever a URL serves straight into the bucket. Get hands back a reference by default (size, type, and a temporary link) and can return small text inline instead. List gives the keys under a prefix. Presign mints a temporary URL: a GET link for somebody to download with, or a PUT slot for somebody to upload into, without either of them holding a key.
Size¶
Bytes do not travel through a run. A step's output is carried along the whole workflow, and it has a hard payload limit, so: - inline text (content to put, or a text get) is capped at 256 KB; - a copy from a source URL is capped at 25 MB and goes straight to the bucket; - anything larger belongs on a presigned PUT URL, which the far side uploads to directly. Each ceiling refuses with a message that says so. Nothing is quietly truncated.
Gotchas¶
- A get returns a link, not the file. That is deliberate: an image inline would break the run at the next step.
- A presigned link is bearer access for as long as it lives. Keep the expiry short.
- A private or local endpoint (MinIO on your own machine) has to be named on the connection with allowPrivateEndpoint, otherwise it is refused as an SSRF risk.
- Most S3-compatible stores need path-style addressing; a connection with a custom endpoint uses it by default.
Configuration reference¶
connectorId(required) - The object store connection to use. It carries the keys, the region, the bucket and, for a store that is not Amazon's, the endpoint.operation(required) - Put a file in, get one back, list what is under a prefix, or presign a temporary link somebody else can use.key- The object's full path inside the bucket, e.g. returns/{{order.data.id}}/photo.jpg. Every operation but List needs one.bucket- Overrides the connection's bucket, so one connection can serve several.body- Put: the text to store. Capped at 256 KB because it travels through the run; use Copy from URL for anything bigger.sourceUrl- Put: fetch this URL and store what it serves, up to 25 MB. The bytes go straight to the bucket and never enter the run's data.contentType- How the object is served later, e.g. image/jpeg. Taken from the source when copying from a URL.output- Reference: size, type and a temporary link, moving no bytes. Text: the content inline, up to 256 KB, for text only.prefix- List: only keys starting with this, e.g. products/2026/.maxKeys- List: how many objects to return, up to 1000.continuationToken- List: the nextToken an earlier List gave back, to read the next page.method- GET hands somebody a download link. PUT hands them an upload slot, which is how a file too big for this step gets into the bucket.expiresIn- How long a presigned link stays valid. 15 minutes by default, 7 days at most.allowedHosts- The hosts Copy from URL may point at. Naming one is what allows a private or local address, which is otherwise refused.
Output fields¶
bucketkeysizecontentTypeetaglastModifiedlocationurlexpiresInexpiresAtmethodinlinebodyprefixobjectscounttruncatednextToken