PowerShell : Set-Content Replace word and Encoding UTF8 without BOM

Question

I'd like to escape \ to \\ in csv file to upload to Redshift. Following simple PowerShell script can replace $TargetWord \ to $ReplaceWord \\ , as expected, but export utf-8 with bom and sometimes causes the Redshift copy error.

Any advice would be appreciated to improve it. Thank you in advance.

Exp_Escape.ps1

Param(
    [string]$StrExpFile,
    [string]$TargetWord,
    [string]$ReplaceWord
)

# $(Get-Content "$StrExpFile").replace($TargetWord,$ReplaceWord) | Set-Content -Encoding UTF8 "$StrExpFile"

Answer 1

• In PowerShell (Core) 7+, you would get BOM-less UTF-8 files by default; -Encoding utf8 and -Encoding utf8NoBom express that default explicitly; to use a BOM, -Encoding utf8BOM is needed.

• In Windows PowerShell, unfortunately, you must directly call .NET APIs to get BOM-less UTF-8, because -Encoding utf8 only produces UTF-8 files with BOM (and no other utf8-related values are supported).

# In order for .NET API calls to work as expected,
# file paths must be expressed as *full, native* paths.
$OutDir = Split-Path -Parent $StrExpFile
if ($OutDir -eq '') { $OutDir = '.' }
$strExpFileFullPath = Join-Path (Convert-Path $OutDir) (Split-Path -Leaf $StrExpFile)

# Note: .NET APIs create BOM-less UTF-8 files *by default*
[IO.File]::WriteAllLines(
  $strExpFileFullPath,
  (Get-Content $StrExpFile).Replace($TargetWord, $ReplaceWord)
)

The above uses the System.IO.File.WriteAllLines method.

For a convenience wrapper function around Out-File for use in Windows PowerShell that creates BOM-less UTF-8 files, see this answer.