Object vs block vs file storage

When someone says "I need storage", they may mean three very different things. A database wants a fast disk. A team wants a shared folder. A website wants to serve thousands of images. Each need calls for its own model, and picking the wrong one costs you in performance or in complexity.

The three models are block storage, file storage and object storage. The difference is who organizes the data and how you reach it.

How does each storage type keep data?#

Block. Data is split into fixed-size blocks, each with an address. You read and write individual blocks directly. The result is a raw device with no folders: the operating system formats it with a file system (ext4, XFS, NTFS) and uses it as a disk. The Amazon EBS documentation describes exactly this: a block device you attach to an instance and use as you would a physical hard drive.

File. Data lives in a hierarchy of folders and files, with names, paths and user and group permissions. The difference from block is that the file system comes ready and can be shared: many servers mount the same folder over the network, usually through NFS or SMB. Amazon EFS, for example, uses NFS 4 and POSIX permissions.

Object. Each file becomes an object: the content, a unique key and metadata. Objects sit in a flat namespace (a bucket) and are reached through an HTTP API with operations such as PUT, GET and DELETE. There is no mounted disk and no real folder. The details are in what object storage is.

What changes in practice?#

The same file, two ways of reaching it:

import os

import boto3


def edit_on_disk():
    # Block or file: the operating system gives you a mounted path.
    # You open the file and write in the middle of it.
    with open("/data/report.csv", "r+b") as f:
        f.seek(1024)
        f.write(b"new content")  # changes only these bytes


def upload_as_object():
    # Object: no mounted path, only a key and an HTTP API.
    # To change one byte, you upload the whole object again.
    s3 = boto3.client(
        "s3",
        endpoint_url=os.environ["S3_ENDPOINT"],
        aws_access_key_id=os.environ["S3_ACCESS_KEY"],
        aws_secret_access_key=os.environ["S3_SECRET_KEY"],
    )
    with open("report.csv", "rb") as f:
        s3.put_object(Bucket="my-bucket", Key="reports/report.csv", Body=f)

In the first case the program uses file system calls and changes bytes in place. In the second, every operation is a network request that reads or replaces the whole object. That is why a program written for a disk does not work on object storage without rework, and the reverse is true too.

Side-by-side comparison#

AWS describes block storage as low latency, file storage as best suited to shared access, and object storage as near-infinitely scalable, with more latency on access. Numbers vary by provider and volume type, so check the documentation of the service you plan to use.

When should you pick each one?#

Pick block when an application needs a fast, dedicated disk: databases, systems with many small reads and writes, and the operating system disk. The volume is usually attached to one server at a time, with exceptions at some providers, such as EBS Multi-Attach.

Pick file when several servers must see the same folder tree at once, with permissions and file system semantics. Examples: an uploads directory shared between replicas of a legacy app, or a media repository edited by a team.

Pick object when you keep many files that are uploaded and downloaded whole: avatars, product images, videos, PDFs, archived logs, exports. It is the best starting point for content that grows without a fixed limit and must be served over the internet. To decide where images go, see images in the database vs object storage.

Combine all three. Real applications mix types: the database runs on block storage, and images uploaded by users go to object storage, with only the key stored in the database. There is no universal winner, only the right type for each piece of data.

Does object storage replace the app's disk?#

Not for everything. If your code opens files by path and writes in the middle of them, it needs a disk or a file system. But if the server disk only holds uploads, moving them to object storage solves three problems at once: you stop running out of space, files survive the server being recreated, and several instances see the same data.

Because the Amazon S3 API became a de facto standard, many providers offer compatible storage. That lets you switch providers by changing, in most cases, the endpoint and the credentials. See what S3-compatible storage is.

How to do this on Square Cloud#

For objects, Square Cloud offers Blob Storage: object storage with an S3-compatible API (aws-cli, rclone, boto3 and the AWS SDKs), plus a REST API and a dashboard. The upload_as_object function above is a starting point: set S3_ENDPOINT to https://s3-blob.squarecloud.app and fill S3_ACCESS_KEY and S3_SECRET_KEY with the keys from the "S3 access" tab of the dashboard.

Files can be up to 10 GB, and multipart is only required above 100 MB. There is no egress or ingress fee, and every plan includes a quota from 5 GB to 250 GB. Public files are served by a global CDN at the edge, and private files open through a temporary link or a share link. You can also set folder rules, such as expiration, cache and allowed extensions.

See how it works on the Blob Storage page.