Skip to content

S3 Storage

Zero-dependency S3-compatible storage client with AWS Signature V4.

Replaces: boto3, minio, s3fs

Overview

A minimal S3 REST client covering common object storage operations. Compatible with AWS S3, Cloudflare R2, Oracle Object Storage, MinIO, and any other S3-compatible backend. Uses only Python stdlib.

File Description Dependencies
s3.py S3 client with SigV4 auth None (stdlib only: hashlib, hmac, http.client, urllib.parse, xml.etree)

Key Features

  • Bucket operations -- bucket_exists, make_bucket, remove_bucket, list_buckets
  • Object operations -- get_object, put_object, stat_object, remove_object, copy_object, list_objects
  • AWS Signature V4 -- standard authentication for S3-compatible services
  • URL styles -- path-style (endpoint/bucket/key) and virtual-host-style (bucket.endpoint/key)
  • Streaming -- chunked downloads via iter_chunks()
  • Multi-provider -- AWS S3, Cloudflare R2, Oracle OCI, MinIO out of the box

How to Use in Your Project

cp s3/s3.py your_project/

Usage Examples

AWS S3

from s3 import S3Client

client = S3Client(
    endpoint="s3.amazonaws.com",
    access_key="AKIA...",
    secret_key="wJal...",
    region="us-east-1",
)

# Upload
import io
data = b"hello world"
client.put_object("my-bucket", "hello.txt", io.BytesIO(data), length=len(data))

# Download
response = client.get_object("my-bucket", "hello.txt")
try:
    print(response.read())
finally:
    response.close()

# List objects
for obj in client.list_objects("my-bucket", prefix="data/"):
    print(obj.key, obj.size)

Cloudflare R2

client = S3Client(
    endpoint="<account>.r2.cloudflarestorage.com",
    access_key="...",
    secret_key="...",
    region="auto",
    url_style="path",
)