An incomplete S3 multipart upload is storage you pay for that is not an object. Every part uploaded before the final complete request is billed as storage, S3 keeps it indefinitely, and the one lifecycle setting that removes it is separate from the expiration rules most buckets already have. An expiration rule on the same prefix does nothing to it.
What is an incomplete multipart upload in S3?
A multipart upload is three steps: create, upload parts, complete. The
S3 multipart upload overview
describes AWS's own example of a 100 GB file as one
CreateMultipartUpload call, 1,000 UploadPart calls of
100 MB each, and one CompleteMultipartUpload, "a total of 1,002
API calls". An upload is incomplete for as long as that last call has not
succeeded and nobody has aborted it.
Nothing times it out. The same page lists it as a feature: "After you initiate a multipart upload, there is no expiry; you must explicitly complete or stop the multipart upload." That is useful for resuming a slow upload and costly for every process that dies halfway through one.
You do not have to call the multipart API yourself to create one. The boto3
upload_fileobj reference
describes it as "a managed transfer which will perform a multipart upload in
multiple threads if necessary". Athena does the same with query results: the
Athena query results documentation
says "Athena uses Amazon S3 multipart uploads to write data Amazon S3", which
is why cancelled and failed queries can leave incomplete uploads behind.
Do incomplete S3 multipart uploads cost money?
Yes, for storage and for the requests already made. The overview is direct: "Throughout its lifetime, you are billed for all storage, bandwidth, and requests for this multipart upload and its associated parts." And it is explicit about when that stops: "Only after you complete or stop a multipart upload will Amazon S3 free up the parts storage and stop billing you for the parts storage."
Take AWS's 1,000-part example and kill the uploader after part 700. There is no object, but 70 GB of parts are stored and 701 requests have been billed. Left alone, that is 70 GB-months of storage every month, or 840 GB-months over a year, for data nobody can read.
Which storage rate applies depends on the storage class the upload named:
| Storage class requested | How in-progress parts are billed |
|---|---|
| S3 Glacier Flexible Retrieval | As Glacier Flexible Retrieval staging storage at S3 Standard rates until the upload completes |
| S3 Glacier Deep Archive | Also as Glacier Flexible Retrieval staging storage at S3 Standard rates |
| Any other storage class, including S3 Standard | At the storage class specified when the parts were uploaded |
The two Glacier rows are the ones that surprise people. AWS says "in-progress
multipart parts for a PUT request to the S3 Glacier Flexible Retrieval storage
class are billed as S3 Glacier Flexible Retrieval staging storage at S3
Standard storage rates until the upload completes", and that
CreateMultipartUpload and UploadPart "are billed at
S3 Standard rates". An abandoned archive upload is therefore billed like hot
storage, not like an archive.
The good news is on the way out: "There are no early delete charges for deleting incomplete multipart uploads regardless of storage class specified." That is the opposite of the rule for real objects, where minimum storage durations bill you for days you did not keep. Aborting never triggers an early delete fee.
Why are incomplete multipart uploads invisible in the S3 console?
Because there is no object to show. The aborting a multipart upload page puts it plainly: "If you don't send the complete multipart upload request successfully, S3 does not assemble the parts and does not create any object." An object listing lists objects, so the parts never appear in one.
AWS's own Cloud Financial Management team states the console gap directly in discovering and deleting incomplete multipart uploads: "you aren't able to view the parts of your incomplete multipart upload in the AWS Management Console." The bucket looks smaller than the bill says it is, and while such uploads exist, summing object sizes will not reconcile to the storage you are charged for.
How do you list incomplete multipart uploads in an S3 bucket?
With ListMultipartUploads, which the
API reference
says covers "a multipart upload that has been initiated by the
CreateMultipartUpload request, but has not yet been completed or aborted".
Each entry carries the key, the
upload ID, the storage class and an Initiated timestamp, which is
the field that tells you how old it is:
aws s3api list-multipart-uploads \
--bucket amzn-s3-demo-bucket \
--query 'Uploads[].[Key,Initiated,StorageClass]' \
--output table
A single API response stops at 1,000 uploads, and further pages need both
key-marker and upload-id-marker. The CLI handles that
for you: the
list-multipart-uploads command reference
notes it "is a paginated operation", so it keeps calling until the set is
complete. Listing needs s3:ListBucketMultipartUploads on the
bucket, a permission separate from ordinary object listing.
The response does not include a size. To see how many bytes one upload is
holding, call
ListParts
for it, which returns a Size for each part (up to 1,000 parts per
response):
aws s3api list-parts \
--bucket amzn-s3-demo-bucket --key path/to/key \
--upload-id EXAMPLE-UPLOAD-ID \
--query 'sum(Parts[].Size)'
To remove one by hand, call abort-multipart-upload with the same
bucket, key and upload ID. Check it worked, because the
AbortMultipartUpload reference
warns that parts still uploading "might or might not succeed. As a result, it
might be necessary to abort a given multipart upload multiple times in order
to completely free all storage consumed by all parts." Its advice is to call
ListParts afterwards and confirm the list is empty.
How does the S3 AbortIncompleteMultipartUpload lifecycle rule work?
It sets a maximum age, counted from initiation. The abort incomplete multipart upload lifecycle configuration guide says an upload not finished in that window "becomes eligible for an abort operation. Amazon S3 then stops the multipart upload and deletes the parts associated with the multipart upload." AWS's CLI example on that page uses seven days; this is the same configuration with only the rule ID changed:
{
"Rules": [
{
"ID": "abort-incomplete-mpu",
"Status": "Enabled",
"Filter": { "Prefix": "" },
"AbortIncompleteMultipartUpload": { "DaysAfterInitiation": 7 }
}
]
}
Four details decide whether it does what you expect:
- It reaches back. "This rule applies to both existing multipart uploads and those that you create later." Adding it clears the backlog, not just future debris.
- It never touches objects. "No objects are deleted by this lifecycle action", and uploads that finish inside the window are left alone. It is about as safe as a lifecycle rule gets.
- It cannot be scoped by tag. The lifecycle configuration examples state: "When you use the AbortIncompleteMultipartUpload S3 Lifecycle action, the rule cannot specify a tag-based filter." Scope it by prefix, or apply it to the whole bucket.
-
It is counted in days from initiation.
DaysAfterInitiationdoes not care whether an upload is still making progress, so an upload that is legitimately running when the window closes is eligible for abort with the rest. The number has to exceed your longest real upload.
Why does an S3 expiration rule not delete incomplete multipart uploads?
Because expiration acts on objects, and these are not objects yet. The lifecycle rules reference says so in a note that is easy to scroll past: "Object expiration lifecycle configurations don't remove incomplete multipart uploads. To remove incomplete multipart uploads, you must use the AbortIncompleteMultipartUpload Lifecycle configuration action".
This is the trap for anyone who has tidied a scratch prefix. A seven-day
expiration on athena-results/ or tmp/ looks like
complete garbage collection, and it clears every finished file. The abandoned
uploads under the same prefix stay, billed, until a separate action removes
them, and the fix is an abort rule covering the same prefix.
Put the earlier hypothetical under that prefix. A 70 GB upload abandoned
under tmp/ in a bucket with only a seven-day expiration rule is
never touched by it: after a year it has billed 840 GB-months, while every
finished file under the prefix was expired on schedule. The same
upload under an AbortIncompleteMultipartUpload rule of seven
days becomes eligible for abort a week after initiation. AWS uses seven days
in its own lifecycle example, and Storage Lens reports incomplete bytes
older than seven days as a separate metric, so the two pages line up on that
number.
What does S3 Storage Lens show about incomplete multipart uploads for free?
The byte and count figures, with a short memory. The Storage Lens metrics glossary lists these in the free tier, under cost optimization:
| Metric | Export name | Tier |
|---|---|---|
| Incomplete multipart upload bytes | IncompleteMultipartUploadStorageBytes | Free |
| Incomplete multipart upload object count | IncompleteMultipartUploadObjectCount | Free |
| Incomplete multipart upload storage bytes greater than 7 days old | IncompleteMPUStorageBytesOlderThan7Days | Free |
| Incomplete multipart upload object count greater than 7 days old | IncompleteMPUObjectCountOlderThan7Days | Free |
| Abort incomplete multipart upload lifecycle rule count | AbortIncompleteMPULifecycleRuleCount | Advanced |
Three things about that table are worth knowing before you rely on it.
The percentage includes the debris in its denominator. The glossary defines total storage as "inclusive of incomplete multipart uploads, object metadata, and delete markers", and the percentage metric divides by it. A bucket holding 930 GB of objects and the 70 GB abandoned upload from earlier reports 7.0%, not the 7.53% you would get dividing by object bytes.
The console forgets free data after two weeks; an export does not. The Storage Lens metrics selection page says "All free tier metrics are collected daily and can be exported to either an S3 general purpose bucket (CSV or Parquet format) or S3 table bucket (Parquet format only). Data is available for queries for 14 days in the Amazon S3 console." To see whether incomplete bytes are growing over months on the free tier, turn on the daily export and keep the files. CloudWatch publishing, which is what an alarm would need, is an advanced-tier feature.
The pages do not agree on whether the rule check is free.
The
Storage Lens overview
says "The default metrics report in S3 Storage Lens includes free metrics and
advanced tier metrics", that it can "identify cost optimization opportunities
like buckets without S3 Lifecycle rules for incomplete multipart uploads", and
that its recommendations come "at no additional charge beyond standard S3
storage costs". The metrics selection page describes the same report as one
that "includes free metrics and prefix aggregation capabilities", and the
glossary lists the metric that counts abort rules,
AbortIncompleteMPULifecycleRuleCount, as Advanced. Read together,
the safe assumption is that counting abort rules is a paid metric. On the
free tier, the practical test is indirect: a bucket
with non-zero bytes in the older-than-7-days metric is the one to check, and
get-bucket-lifecycle-configuration on it shows whether any rule
carries AbortIncompleteMultipartUpload, and for which prefix.
What else inflates an S3 data lake bill besides incomplete multipart uploads?
Incomplete uploads are one of several S3 and query costs that do not show up where you look first:
- Athena's per-query and per-workgroup limits explain why a cancelled query still bills for the bytes it scanned, as well as leaving partial results behind.
- S3 storage class minimums and the 128 KB rule cover the charges a lifecycle transition can add, which is the other half of writing lifecycle rules safely.
-
What AWS KMS actually costs
matters for multipart uploads to SSE-KMS buckets, where the multipart
permissions table in the overview requires
kms:GenerateDataKeyforCreateMultipartUploadandkms:DecryptforUploadPart. - Athena partition projection over the CUR shows how CUR data is laid out in S3 prefix by prefix, which is also the shape abort rules are scoped by.
The parts themselves are billed as storage, but every operation that created them was billed as a request, and multipart uploads are charged at S3 Standard request rates even when the destination is an archive class: S3 request pricing by class.