I have below code to get the list of objects in the bucket, but it does not work for the sub folders of the S3.

import boto3

bucket_from = "BucketSource"
s3 = boto3.resource('s3')
src = s3.Bucket(bucket_from)

for archive in src.objects.all():
    print(archive.key)

While using sub folders, FirstLevelS3/SecondLevelS3/BucketSource, this is the error:

Bucket name must match the regex "^[a-zA-Z0-9.\-_]

I have below code to get the list of objects in the bucket, but it does not work for the sub folders of the S3.

import boto3

bucket_from = "BucketSource"
s3 = boto3.resource('s3')
src = s3.Bucket(bucket_from)

for archive in src.objects.all():
    print(archive.key)

While using sub folders, FirstLevelS3/SecondLevelS3/BucketSource, this is the error:

Bucket name must match the regex "^[a-zA-Z0-9.\-_]

• python
• amazon-web-services
• amazon-s3
• boto3

Share Improve this question Follow edited Mar 21 at 20:31 John Rotenstein 271k2828 gold badges447447 silver badges531531 bronze badges Recognized by AWS Collective asked Mar 21 at 12:33 Jim MacaulayJim Macaulay 5,18955 gold badges3232 silver badges5959 bronze badges 3

• 1 This question is similar to: S3 listing all files in subfolder in a bucket. If you believe it’s different, please edit the question, make it clear how it’s different and/or how the answers on that question are not helpful for your problem. – aneroid Commented Mar 21 at 15:18
• 1 TL;DR from the linked duplicate: You need to use the Prefix parameter: for archive in src.objects.filter(Prefix='FirstLevelS3/SecondLevelS3/'): ... – aneroid Commented Mar 21 at 15:20
• 2 What do you mean by "While using sub folders"? Are you saying that the code as shown generates the error? I can't see where you are attempting to specify the folders. – John Rotenstein Commented Mar 21 at 20:35

Add a comment  | 

2 Answers 2

Sorted by: Reset to default 2

It sounds like you are trying to list S3 objects under a given key prefix and to do that you have accidentally provided the key prefix as a suffix on the bucket name, for example:

src = s3.Bucket('mybucket/myphotos/dogs/')

for obj in src.objects.all():
    print(obj.key)

That fails because you have supplied a bucket name of mybucket/myphotos/dogs/ which indeed is not legal for a bucket name.

You should instead only use the bucket name e.g. mybucket when calling s3.Bucket(...) and then filter the objects on the desired key prefix, for example:

src = s3.Bucket('mybucket')

for obj in src.objects.filter(Prefix='myphotos/dogs/'):
    print(obj.key)

the error message "Bucket name must match the regex '^[a-zA-Z0-9.\-_]'" is indicating that your bucket name, "Bucket_Source", contains an underscore (_), which is not allowed in an S3 bucket name