Wasabi File Upload via Powershell S3 - Stack Overflow

I am facing the issue that I encounter the following error:"The request body terminated unexpected

I am facing the issue that I encounter the following error:"The request body terminated unexpectedly" Wrote to the wasabi support they told me that in the logs they encounter an 410 error. In Powershell it self the upload goes to 100% then it gets terminated, my code at the moment looks like this:


$SiteUrl = "sharepoint site"    
$LibraryName = "Documents"
$FolderName = "Archive"
$S3BucketName = "Bucket"    
$S3Region = "eu-central-2" 
$AccessKey = "Access_Key" 
$SecretKey = "Secret_Key" 

$LocalTempFolder = "Folderpath"  
$WasabiEndpoint = ";

Import-Module PnP.PowerShell
Import-Module AWSPowerShell.NetCore


function Convert-HexStringToByteArray {
    param (
        [string]$HexString
    )
    
    
    if ($HexString.Length % 2 -ne 0) {
        throw "Hex string length must be even."
    }

    
    $ByteArray = for ($i = 0; $i -lt $HexString.Length; $i += 2) {
        [Convert]::ToByte($HexString.Substring($i, 2), 16)
    }
    
    return $ByteArray
}

function Upload-ToWasabi {
    param (
        [string]$FilePath,
        [string]$S3Key,
        [int]$RetryCount = 3
    )

    for ($Attempt = 1; $Attempt -le $RetryCount; $Attempt++) {
        try {
            Write-Host "Uploading item to Wasabi S3: $S3Key (Attempt $Attempt of $RetryCount)"

            
            Write-S3Object -BucketName $S3BucketName `
               -Key $S3Key `
               -File $FilePath `
               -Region $S3Region `
               -ProfileName WasabiProfile `
               -Endpoint $WasabiEndpoint `
               -StorageClass STANDARD `
               -PartSize 5MB `
               -Verbose

            Write-Host "Upload successful for $S3Key."
            return $true
        } catch {
            Write-Host "Error during upload: $($_.Exception.Message)"
            Write-Host "Full error details: $($_.Exception)"
            if ($Attempt -lt $RetryCount) {
                Write-Host "Retrying upload in 5 seconds..."
                Start-Sleep -Seconds 5
            } else {
                Write-Error "Failed to upload $S3Key after $RetryCount attempts."
                return $false
            }
        }
    }
}





Write-Host "Connecting to SharePoint..."
Connect-PnPOnline -Url $SiteUrl -UseWebLogin


if (-not (Test-Path -Path $LocalTempFolder)) {
    Write-Host "Creating temporary folder at $LocalTempFolder"
    New-Item -Path $LocalTempFolder -ItemType Directory
}

$SiteName = ($SiteUrl.Split('/'))[-1]  
Write-Host "Using SharePoint site name: $SiteName"

try {
    
    Write-Host "Fetching the SharePoint library and folder..."
    $FolderRelativeUrl = "Shared Documents/$FolderName"
    
    Write-Host "Getting files from SharePoint folder: $FolderRelativeUrl"
    $Files = Get-PnPFolderItem -FolderSiteRelativeUrl $FolderRelativeUrl

    
    Write-Host "Items found in SharePoint folder: $($Files.Count)"
    foreach ($Item in $Files) {
        Write-Host "Processing item: $($Item.Name) - Type: $($Item.FileSystemObjectType)"
        
        
        $LocalFilePath = Join-Path -Path $LocalTempFolder -ChildPath $Item.Name

        try {
            
            Write-Host "Downloading item: $($Item.Name) to $LocalFilePath"
            Get-PnPFile -Url $Item.ServerRelativeUrl -Path $LocalTempFolder -FileName $Item.Name -AsFile -Force

            
            if (Test-Path $LocalFilePath) {
                $FileSize = (Get-Item $LocalFilePath).Length
                Write-Host "Item downloaded successfully: $LocalFilePath"
                Write-Host "Item size: $FileSize bytes"

                
                $S3Key = "$SiteName/$FolderName/$($Item.Name)"

                
                $UploadSuccess = Upload-ToWasabi -FilePath $LocalFilePath -S3Key $S3Key

                if (-not $UploadSuccess) {
                    Write-Error "Failed to upload item to Wasabi: $S3Key"
                }
            } else {
                Write-Host "Error: Item was not downloaded properly: $LocalFilePath"
            }
        } catch {
            Write-Host "Error processing item: $($Item.Name)"
            Write-Error $_
        }
    }
} catch {
    Write-Error "An error occurred: $_"
} finally {
    Write-Host "Disconnecting from SharePoint..."
    Disconnect-PnPOnline
}

Write-Host "Script completed successfully."

I was trying to download files from sharepoint which works and upload them to wasabi to a specific bucket and also a folder in it, the user has s3 full access.

I am facing the issue that I encounter the following error:"The request body terminated unexpectedly" Wrote to the wasabi support they told me that in the logs they encounter an 410 error. In Powershell it self the upload goes to 100% then it gets terminated, my code at the moment looks like this:


$SiteUrl = "sharepoint site"    
$LibraryName = "Documents"
$FolderName = "Archive"
$S3BucketName = "Bucket"    
$S3Region = "eu-central-2" 
$AccessKey = "Access_Key" 
$SecretKey = "Secret_Key" 

$LocalTempFolder = "Folderpath"  
$WasabiEndpoint = "https://s3.eu-central-2.wasabisys"

Import-Module PnP.PowerShell
Import-Module AWSPowerShell.NetCore


function Convert-HexStringToByteArray {
    param (
        [string]$HexString
    )
    
    
    if ($HexString.Length % 2 -ne 0) {
        throw "Hex string length must be even."
    }

    
    $ByteArray = for ($i = 0; $i -lt $HexString.Length; $i += 2) {
        [Convert]::ToByte($HexString.Substring($i, 2), 16)
    }
    
    return $ByteArray
}

function Upload-ToWasabi {
    param (
        [string]$FilePath,
        [string]$S3Key,
        [int]$RetryCount = 3
    )

    for ($Attempt = 1; $Attempt -le $RetryCount; $Attempt++) {
        try {
            Write-Host "Uploading item to Wasabi S3: $S3Key (Attempt $Attempt of $RetryCount)"

            
            Write-S3Object -BucketName $S3BucketName `
               -Key $S3Key `
               -File $FilePath `
               -Region $S3Region `
               -ProfileName WasabiProfile `
               -Endpoint $WasabiEndpoint `
               -StorageClass STANDARD `
               -PartSize 5MB `
               -Verbose

            Write-Host "Upload successful for $S3Key."
            return $true
        } catch {
            Write-Host "Error during upload: $($_.Exception.Message)"
            Write-Host "Full error details: $($_.Exception)"
            if ($Attempt -lt $RetryCount) {
                Write-Host "Retrying upload in 5 seconds..."
                Start-Sleep -Seconds 5
            } else {
                Write-Error "Failed to upload $S3Key after $RetryCount attempts."
                return $false
            }
        }
    }
}





Write-Host "Connecting to SharePoint..."
Connect-PnPOnline -Url $SiteUrl -UseWebLogin


if (-not (Test-Path -Path $LocalTempFolder)) {
    Write-Host "Creating temporary folder at $LocalTempFolder"
    New-Item -Path $LocalTempFolder -ItemType Directory
}

$SiteName = ($SiteUrl.Split('/'))[-1]  
Write-Host "Using SharePoint site name: $SiteName"

try {
    
    Write-Host "Fetching the SharePoint library and folder..."
    $FolderRelativeUrl = "Shared Documents/$FolderName"
    
    Write-Host "Getting files from SharePoint folder: $FolderRelativeUrl"
    $Files = Get-PnPFolderItem -FolderSiteRelativeUrl $FolderRelativeUrl

    
    Write-Host "Items found in SharePoint folder: $($Files.Count)"
    foreach ($Item in $Files) {
        Write-Host "Processing item: $($Item.Name) - Type: $($Item.FileSystemObjectType)"
        
        
        $LocalFilePath = Join-Path -Path $LocalTempFolder -ChildPath $Item.Name

        try {
            
            Write-Host "Downloading item: $($Item.Name) to $LocalFilePath"
            Get-PnPFile -Url $Item.ServerRelativeUrl -Path $LocalTempFolder -FileName $Item.Name -AsFile -Force

            
            if (Test-Path $LocalFilePath) {
                $FileSize = (Get-Item $LocalFilePath).Length
                Write-Host "Item downloaded successfully: $LocalFilePath"
                Write-Host "Item size: $FileSize bytes"

                
                $S3Key = "$SiteName/$FolderName/$($Item.Name)"

                
                $UploadSuccess = Upload-ToWasabi -FilePath $LocalFilePath -S3Key $S3Key

                if (-not $UploadSuccess) {
                    Write-Error "Failed to upload item to Wasabi: $S3Key"
                }
            } else {
                Write-Host "Error: Item was not downloaded properly: $LocalFilePath"
            }
        } catch {
            Write-Host "Error processing item: $($Item.Name)"
            Write-Error $_
        }
    }
} catch {
    Write-Error "An error occurred: $_"
} finally {
    Write-Host "Disconnecting from SharePoint..."
    Disconnect-PnPOnline
}

Write-Host "Script completed successfully."

I was trying to download files from sharepoint which works and upload them to wasabi to a specific bucket and also a folder in it, the user has s3 full access.

Share Improve this question asked Jan 29 at 11:32 insilinsil 1 1
  • see stackoverflow/questions/74103581/… – jdweng Commented Jan 29 at 13:17
Add a comment  | 

1 Answer 1

Reset to default 0

I found the Solution with the Wasabi support and the fault is the newer AWSPowershell.NetCore --> Derzeit unterstützt der Wasabi Endpoint nur Version: AWSPowerShell.NetCore 4.1.736. Wasabi sucht derzeit auch nach einer Lösung für die Behebung.

发布者:admin,转转请注明出处:http://www.yc00.com/questions/1745301497a4621461.html

相关推荐

  • Wasabi File Upload via Powershell S3 - Stack Overflow

    I am facing the issue that I encounter the following error:"The request body terminated unexpected

    7小时前
    10

发表回复

评论列表(0条)

  • 暂无评论

联系我们

400-800-8888

在线咨询: QQ交谈

邮件:admin@example.com

工作时间:周一至周五,9:30-18:30,节假日休息

关注微信