I am writing a Python script using python-swiftclient and zipfile to zip and upload files to the Swift API endpoint of an Openstack Object Store. I store the zipped data in memory as an io.BytesIO object.

Code snippet:

arc_name = 'test.zip'
zip_buffer = io.BytesIO()
with open zipfile.ZipFile(zip_buffer, "a", zipfile.ZIP_DEFLATED, True) as zip_file:
    for file in files:
        with open(file, 'rb') as src_file:
            zip_file.writestr(arc_name, src_file.read())

...

zip_data = zip_buffer.getvalue()
checksum_base64 = base64.b64encode(hashlib.md5(zip_data).digest()).decode()

swift_conn = swiftclient.Connection(<creds>)
container_name = 'swift-test'
swift_conn.put_object(container=container_name, contents=zip_data, content_type=None, obj=arc_name, etag=checksum_base64)

The error is:

swiftclient.exceptions.ClientException: Object PUT failed: https://..../swift/v1/swift-test/test.zip 422 Unprocessable Entity

From other HTTP 422 error questions my thinking is the issue is content_type (MIME type). I've tried 'application/zip' and 'multipart/mixed' but always see the same error.

If another MIME type is more appropriate, or I'm missing something else, I'd be grateful for any help.

I am writing a Python script using python-swiftclient and zipfile to zip and upload files to the Swift API endpoint of an Openstack Object Store. I store the zipped data in memory as an io.BytesIO object.

Code snippet:

arc_name = 'test.zip'
zip_buffer = io.BytesIO()
with open zipfile.ZipFile(zip_buffer, "a", zipfile.ZIP_DEFLATED, True) as zip_file:
    for file in files:
        with open(file, 'rb') as src_file:
            zip_file.writestr(arc_name, src_file.read())

...

zip_data = zip_buffer.getvalue()
checksum_base64 = base64.b64encode(hashlib.md5(zip_data).digest()).decode()

swift_conn = swiftclient.Connection(<creds>)
container_name = 'swift-test'
swift_conn.put_object(container=container_name, contents=zip_data, content_type=None, obj=arc_name, etag=checksum_base64)

The error is:

swiftclient.exceptions.ClientException: Object PUT failed: https://..../swift/v1/swift-test/test.zip 422 Unprocessable Entity

From other HTTP 422 error questions my thinking is the issue is content_type (MIME type). I've tried 'application/zip' and 'multipart/mixed' but always see the same error.

If another MIME type is more appropriate, or I'm missing something else, I'd be grateful for any help.

• python
• mime-types
• python-zipfile
• object-storage
• openstack-swift

Share Improve this question Follow asked Nov 20, 2024 at 13:11 DaveDave 46077 silver badges1313 bronze badges

Add a comment  | 

1 Answer 1

Sorted by: Reset to default 0

I found the issue, and content_type was a red herring. The issue was the etag.

checksum_hash = hashlib.md5(zip_data)
checksum_string = checksum_hash.hexdigest()
checksum_base64 = base64.b64encode(checksum_hash.digest()).decode()

swift_conn.put_object(container=container_name,
    contents=zip_data,
    content_type=None,
    obj=arc_name,
    etag=checksum_base64) # fails

swift_conn.put_object(container=container_name,
    contents=zip_data,
    content_type=None,
    obj=arc_name,
    etag=checksum_string) # works

Also works with content_type='multipart/mixed', which is recommended with zip archives. So the upload was simply being rejected because of mismatched checksums.

The reason I erroneously used checksum_base64 was that coming from boto3 previously, the put_object function call used ContentMD5=checksum_base64.