datalake:seaweedfs

SeaweedFS Object Storage

  • Host: strg2.lan (TrueNAS Scale)
  • Deployment: runs as a TrueNAS app (Docker-based)
  • Network: internal LAN access only — not reachable from outside
  • S3 API endpoint: https://strg2.lan:30304 (self-signed TLS cert)

The S3 gateway is the supported way to read/write data. The internal filer port (30303) is not published to the LAN.

Access keys are managed inside the SeaweedFS Admin UI.

  1. Open http://strg2.lan:30300 in a browser and log in with your admin account.
  2. Navigate to the Object StoreUsers section (the exact wording may vary by version — look for S3 / IAM identities).
  3. Either pick an existing identity or click Add User to create a new one. Assign the buckets/permissions it needs.
  4. Copy the Access Key ID and Secret Access Key. The secret is usually shown only once at creation — save it immediately to a password manager.
⚠ Warning: Treat the secret like a password. Anyone with both keys can read/write every bucket the identity has access to.

1. Install a current rclone

The Debian/Pop!_OS apt package is years out of date and missing the SeaweedFS provider plus modern S3 flags. Always install from upstream:

sudo apt remove rclone -y
curl https://rclone.org/install.sh | sudo bash
hash -r
rclone version   # should be 1.65 or newer, with no "-DEV" suffix

2. Create a credentials env file

Keep secrets out of the rclone config so the config can be shared/committed. Create ~/.config/seaweed.env:

export AWS_ACCESS_KEY_ID="your-access-key-here"
export AWS_SECRET_ACCESS_KEY="your-secret-key-here"
export RCLONE_NO_CHECK_CERTIFICATE=true

Lock it down and load it:

chmod 600 ~/.config/seaweed.env
source ~/.config/seaweed.env

To load automatically in every new terminal, append to ~/.bashrc:

[ -f ~/.config/seaweed.env ] && source ~/.config/seaweed.env
ℹ Note: RCLONE_NO_CHECK_CERTIFICATE=true is required because the server's TLS cert is self-signed for localhost. The s3 backend has no per-remote config option for this — only the global flag, which is set via this env var. Drop it once the cert is reissued with proper SAN entries.

3. Add the remote to rclone.conf

Edit ~/.config/rclone/rclone.conf and add:

[seaweed]
type = s3
provider = SeaweedFS
env_auth = true
endpoint = https://strg2.lan:30304
region = us-east-1
force_path_style = true

env_auth = true tells rclone to read AWS_ACCESS_KEY_ID / AWS_SECRET_ACCESS_KEY from the environment. force_path_style = true is required because SeaweedFS doesn't use wildcard DNS for virtual-hosted bucket URLs.

4. Verify

rclone lsd seaweed:

If you see your bucket list, you're done. If it hangs or errors, run with -vv to see what's happening.

rclone ships with completion scripts for bash, zsh, and fish. After setup, <TAB> completes subcommands, flags, and configured remote names — same experience as mc.

System-wide install (bash)

sudo rclone completion bash | sudo tee /etc/bash_completion.d/rclone > /dev/null

Open a new terminal afterward (or source /etc/bash_completion.d/rclone in the current one). Test with:

rclone <TAB><TAB>        # lists subcommands
rclone copy --<TAB>      # lists flags
rclone lsd <TAB>         # lists configured remotes

User-only install (no sudo)

mkdir -p ~/.local/share/bash-completion/completions
rclone completion bash > ~/.local/share/bash-completion/completions/rclone

Live remote-path completion (optional)

Add this to ~/.bashrc to tab-complete bucket and folder names on the server:

complete -o nospace -C rclone rclone

Each <TAB> triggers a network call, so it's a bit slower — but lets you do things like rclone ls seaweed:my<TAB> and have it list actual bucket names.

For zsh or fish, swap bash for zsh / fish in the commands above and place the output in the matching completion directory for your shell.

Listing

# list all buckets
rclone lsd seaweed:
 
# list files in a bucket (recursive by default in rclone)
rclone ls seaweed:mybucket
 
# list a specific subfolder
rclone ls seaweed:mybucket/path/to/folder
 
# tree view
rclone tree seaweed:mybucket
 
# human-readable sizes + summary
rclone size seaweed:mybucket

Uploading

# single file
rclone copy ./report.pdf seaweed:mybucket/reports/
 
# whole directory (recursive, parallel, with progress)
rclone copy ./my-directory seaweed:mybucket/destination/ --progress --transfers 8
 
# sync — only transfer changed/new files, delete on destination if removed locally
rclone sync ./my-directory seaweed:mybucket/destination/ --progress
 
# dry run (preview what would change, no actual transfer)
rclone sync ./my-directory seaweed:mybucket/destination/ --dry-run -v
💡 Tip: Use copy to add/update files without deleting anything on the server. Use sync to make the destination identical to the source — it will delete files on the server that aren't in your local directory.

Downloading

# single file to current directory
rclone copy seaweed:mybucket/path/file.txt ./
 
# whole directory
rclone copy seaweed:mybucket/path/ ./local-folder/ --progress
 
# print a file's content to stdout (no local copy)
rclone cat seaweed:mybucket/path/file.txt
 
# mirror a bucket locally (will delete local files not on server)
rclone sync seaweed:mybucket/ ./local-mirror/ --progress

Deleting

# single file
rclone delete seaweed:mybucket/path/file.txt
 
# all files under a path (keeps the "directory")
rclone delete seaweed:mybucket/old-folder/
 
# remove path entirely, files + structure
rclone purge seaweed:mybucket/old-folder/

Bucket management

# create a bucket
rclone mkdir seaweed:newbucket
 
# delete an empty bucket
rclone rmdir seaweed:oldbucket
  • --progress — live transfer progress bar
  • --transfers N — parallel transfers (default 4, bump to 8–16 for many small files)
  • --checkers N — parallel metadata checks (default 8)
  • --dry-run — preview without doing anything; pair with -v
  • -vv — verbose debug output, essential for troubleshooting
  • --exclude "pattern" — skip matching files (e.g. --exclude "*.tmp")

The same AWS_ACCESS_KEY_ID / AWS_SECRET_ACCESS_KEY env vars work with any S3-compatible tool. Examples assuming --no-verify-ssl or equivalent:

For GUI access on Linux, rclone mount is the recommended option — don't use s3fs-fuse as it deadlocks under concurrent writes.

Symptom Likely cause Fix
Command hangs for ~60s then errors with x509: certificate is valid for localhost TLS cert verification Ensure RCLONE_NO_CHECK_CERTIFICATE=true is exported in the current shell
InvalidAccessKeyId / SignatureDoesNotMatch Bad or unloaded credentials echo $AWS_ACCESS_KEY_ID to confirm env is set; re-source the env file
Connection refused on port 30303 Filer port is not published to LAN Use the S3 gateway on 30304 instead
Unknown flag --s3-no-check-certificate rclone too old Upgrade rclone (see Install step above)
force_path_style errors / wrong-bucket responses DNS / virtual-hosted style Confirm force_path_style = true is in [seaweed] block
  • datalake/seaweedfs.txt
  • Last modified: 2026/05/22 15:02
  • by fabricio