3

I have this code in my powershell script and it doesn't do well on the special characters parts. 

 $request = 'http://151.80.109.18:8082/vrageremote/v1/session/players'
 $a = Invoke-WebRequest -ContentType "application/json; charset=utf-8" $request |
 ConvertFrom-Json    |
 Select -expand Data |
 Select -expand players |
 Select displayName, factionTag | Out-file "$scriptPath\getFactionTag.txt"

In my output file I only get '????' for any special characters. Does anyone know how I can get it to show special characters in my output file?

mklement0
  • 312,089
  • 56
  • 508
  • 622
user1425462
  • 47
  • 2
  • 2
  • 6

3 Answers3

8

Peter Schneider's helpful answer and Nas' helpful answer both address one problem with your approach: You need to:

  • either: access the .Content property on the response object returned by Invoke-WebRequest to get the actual data returned (as a JSON string), which you can then pass to ConvertFrom-Json.

  • or: use Invoke-RestMethod instead, which returns the data directly and parses it into custom objects, so you can work with these objects directly, without the need for ConvertTo-Json; however, with a character-encoding problem such as in this case this is not an option, because explicit re-encoding of the JSON string is needed - see below.

However, you still have a character-encoding problem, because, in the absence of charset information in the response header, PowerShell interprets the UTF-8-encoded JSON string returned as ISO-8859-1-encoded (still applies as of PowerShell 7.0).

There are two possible solutions:

  • Preferably, amend the web service to include charset=utf-8 in the response header's ContenType field.

  • If you can't do that, you must explicitly re-encode the string received to correct the character-encoding misinterpretation.

Here's the implementation of the latter:

$request = 'http://151.80.109.18:8082/vrageremote/v1/session/players'
$a = Invoke-WebRequest -ContentType "application/json; charset=utf-8" $request

# $a.Content now contains the *misinterpreted* JSON string, so we must 
# obtain its byte representation and re-interpret the bytes as UTF-8.
# Encoding 28591 represents the ISO-8859-1 encoding - see https://docs.microsoft.com/en-us/windows/desktop/Intl/code-page-identifiers
$jsonCorrected = [Text.Encoding]::UTF8.GetString(
  [Text.Encoding]::GetEncoding(28591).GetBytes($a.Content)
)

# Now process the reinterpreted string.
$jsonCorrected |
  ConvertFrom-Json    |
  Select -expand Data |
  Select -expand players |
  Select displayName, factionTag | Out-file "$scriptPath\getFactionTag.txt"
mklement0
  • 312,089
  • 56
  • 508
  • 622
3

Use Invoke-RestMethod if you only need the json data without the ParsedHtml, Headers and other objects returned by Invoke-WebRequest

$request = 'http://151.80.109.18:8082/vrageremote/v1/session/players'
$a = Invoke-RestMethod -ContentType "application/json; charset=utf-8" $request |
Select -expand Data |
Select -expand players |
Select displayName, factionTag | Out-file "$scriptPath\getFactionTag.txt"
Nas
  • 1,223
  • 4
  • 7
  • Same result as the one above, This gets rid of the '????' characters but now I'm left with lots of 'Ð'. Like 'УРА' becomes 'УРÐ' – user1425462 Oct 28 '18 at 16:52
  • That's helpful advice in general, but the problem here is one of character encoding, so there is a need to perform explicit re-encoding of the _string_ data received before further processing, so in this special case working with the `.Content` property of the `Invoke-WebRequest` response is required. – mklement0 Oct 28 '18 at 18:08
  • I was trying -ContentType "application/json; charset=utf-8" in header object for my `Invoke-RestMethod` call and it didn't work , passing it like a separate parameter to `Invoke-RestMethod -ContentType "application/json; charset=utf-8" worked perfectly – Shrikant Prabhu Oct 26 '21 at 21:21
2

Try to convert the value of the .Content property to JSON:

$request = 'http://151.80.109.18:8082/vrageremote/v1/session/players'
$a = Invoke-WebRequest -ContentType "application/json; charset=utf-8" $request

($a.Content | convertfrom-json).Data.Players | select DisplayName,FactionTag | Out-file "$scriptPath\getFactionTag.txt" -Encoding Default
mklement0
  • 312,089
  • 56
  • 508
  • 622
Peter Schneider
  • 2,763
  • 1
  • 13
  • 16