eu-west-1 cannot pass rclone’s checksum verification, so
DataRaven sets ignore_checksum automatically on transfers that touch that region (see
below).
At a Glance
s3_no_check_bucket is on by default because CreateBucket only exists in us-east-1 — in
eu-west-1 buckets are created through the Fil One dashboard — and is an explicit key permission
even where it exists. Skipping rclone’s bucket-existence check avoids a spurious failure against a
bucket that is already there.
Credentials
Create a secret with the Fil One provider type. The auth method isfilone_access_key, which requires:
Keys are created in the Fil One dashboard and can be scoped to specific buckets and permissions —
worth doing for a transfer key that only needs one bucket. Note that
CreateBucket and
DeleteBucket are explicit permissions a key must be granted; a default key cannot create
buckets even in us-east-1.
Location Setup
A bucket’s region is fixed when the bucket is created and data cannot be moved between regions
afterwards, so the location’s region simply matches where the bucket lives. Fil One requires
path-style addressing and HTTPS — the configuration DataRaven generates uses both, so there
is nothing to set up.
Example:
Provider Quirks
Bucket naming — no dots
The full rules:- 3–63 characters
- Lowercase letters, numbers, and hyphens only — no dots
- Must begin and end with a letter or number
- Unique within Fil One
us-east-1 accepted everything S3 does — keys with consecutive slashes (//), trailing slashes
on objects with content, Unicode, and 250+ character paths all copied in unchanged. eu-west-1
rejects keys containing consecutive slashes with a 400 (Object name contains unsupported characters), so a transfer carrying such keys fails on those objects mid-run — check source
keys before a migration into eu-west-1.
eu-west-1: checksum verification fails without ignore_checksum
us-east-1 returns standard MD5 ETags and verifies checksums cleanly — no flag needed there.
Large objects uploaded via multipart are unaffected in both regions, because multipart ETags are
never plain MD5s and rclone skips that comparison automatically.
DataRaven sets Fil One remotes to ListObjectsV2
Fil One has no dedicated rclone backend, so DataRaven configures it asprovider = Other.
rclone picks the S3 listing API from that provider string. Other resolves to
ListObjects (V1). The Fil One S3 compatibility page lists that API as a known risk in
eu-west-1.
The risk comes from the way V1 continues a listing. V1 continues with a marker= parameter.
The value of that parameter is a plain object key. In eu-west-1, that key carries a
server-internal suffix. The shape of the suffix changes at each page boundary:
ContinuationToken instead. That token is an opaque
base64 string. Nothing between rclone and the service reads it as a key, so the suffix cannot
stop the listing.
We did not reproduce a failure. Under V1, a listing of 2,001 objects across three pages
completed correctly. This risk is latent, not an observed failure. DataRaven sets V2 because
Fil One documents the risk, and because V2 returns the same keys.
list_version = 2 into every Fil One remote. We verified both
regions with 2,001 objects:
- Each request carries
list-type=2. - Each new page uses
continuation-token. - V2 returns the same key set as V1.
us-east-1 returns bare keys as tokens, so that region does not have this risk. DataRaven
sets V2 in both regions, so the two regions keep the same config.
DataRaven writes this value into the remote, not into the task config. The
--s3-list-version flag in rclone is backend-global. rclone applies one value to every S3
remote in the transfer. This rule includes the remote on the other side of the transfer.
Some S3-compatible endpoints return all pages on V1, but fail on V2. A value set on the task
therefore applies V2 to the remote on the other side of the transfer. That remote can fail on
V2. The config file keeps V2 on the Fil One remote alone.
This design has two consequences:
- V2 applies to every Fil One task, including tasks created before this change. DataRaven builds the remote again on each run, so there is no older task to correct.
- V2 does not appear in the task config. The detail page of the task shows no
s3_list_versionvalue. That field stays empty until you set it.
s3_list_version: 0 on the task. A value set on
the task has priority, because the command-line flag has priority over the config file.
The two regions differ in API surface
us-east-1 supports the larger slice of the S3 API. In eu-west-1:
- Buckets are created and deleted in the dashboard only —
CreateBucketandDeleteBucketare not available via the API (this is whys3_no_check_bucketis a task default). - ETags are not content MD5s — see the checksum warning above; DataRaven defaults
ignore_checksumon for transfers touching this region. - Keys with consecutive slashes (
//) are rejected at upload withObject name contains unsupported characters;us-east-1accepts them. CopyObjectis not confirmed. Cloud-to-cloud transfers into Fil One never need it, but rclone uses it to rewrite an object’s metadata in place — updating only a modification time on an otherwise unchanged object. If a sync keeps erroring on objects whose content hasn’t changed, this is the likely cause; settingsize_onlyin the task config compares by size alone and sidesteps the metadata rewrite.- Missing resources can return
AccessDenied(403) instead of 404. When troubleshooting, a permission error againsteu-west-1may actually mean a mistyped bucket or path. - V1
ListObjectsmarkers carry internal state that changes — see the listing section above. DataRaven setslist_version = 2on the Fil One remote.
Versioning and object lock are set at bucket creation — permanently
Versioning is opt-in when the bucket is created and can never be suspended afterwards. Object lock (Governance or Compliance mode, with a default retention of 1 day to 100 years) requires versioning and is likewise fixed at creation. Two implications for transfer tasks:- On a versioned destination, every overwrite is a new full copy. A sync task that repeatedly rewrites changed objects grows the version stack, and every version is billed as stored data. Deletes place a delete marker; prior versions remain and remain billed.
- Locked versions cannot be overwritten or deleted. On a bucket with object lock, sync
operations that would replace or prune an object inside its retention window fail with
AccessDenied— expected behavior, not a credentials problem.
Performance Notes
Fil One publishes no numeric rate limits.SlowDown (503) responses are throttling, not
failure — rclone backs off and retries automatically; if they persist in execution logs, lower
transfers in the task’s rclone configuration.
For large objects, throughput scales with multipart settings: raise s3_chunk_size (e.g.,
64M) and s3_upload_concurrency in the task’s rclone configuration rather than file-level
transfers. Parts can be 5 MB–5 GB with up to 10,000 parts per object, so any object up to the
5 TB maximum is reachable with room to spare.
Fil One does not expire incomplete multipart uploads. If a large-object transfer is
interrupted partway, the already-uploaded parts remain in the bucket — invisible in listings
but billed as storage — until explicitly aborted. Fil One’s
multipart upload guide shows how to list and
abort stale uploads with the AWS CLI.
What Transfers — and What Doesn’t
Fil One stores object data and standard metadata, and supports more of the S3 feature set than most S3-compatible providers — but not all of it (see the compatibility matrix). Migrating from S3:Importing from rclone.conf
Fil One has noprovider = value of its own in rclone, so real configs write provider = Other.
The importer recognizes Fil One remotes by their endpoint hostname (*.s3.fil.one) instead of
falling back to S3 Compatible, and a config that carries only the endpoint is fine — the region
is recovered from the hostname. Because access keys are region-scoped, each imported remote maps
cleanly to one region’s secret. See the rclone import guide.