JSON.parse(): SyntaxError: Unexpected token � in JSON at position 0

Question

I have a json object which returns from the following PowerShell command:

Get-Service -Name "name" | ConvertTo-Json -Compress > "/to/path/name.json"

If i basically open the file in vscode it seems to be correctly formatted. After i read the file with

fs.readFile(file, 'utf8', (err, data)...

and then try JSON.parse(data) im receiving the error:

undefined:1
��{
^

SyntaxError: Unexpected token � in JSON at position 0

Then i tried to do the following: data.replace(/[^\x00-\x7F]/g, "") to only have ASCII characters, which basically seems to work at least with a console.log().

But JSON.parse then complains:

undefined:1
{
 

SyntaxError: Unexpected token  in JSON at position 1

Im not sure whats the problem there. Hopefully somebody can help me with that.

This is an example json file: As i think the format is correct. Only too much spaces which are removed by the -Compress PowerShell parameter.

{
    "CanPauseAndContinue":  false,
    "CanShutdown":  false,
    "CanStop":  false,
    "DisplayName":  "OpenSSH Authentication Agent",
    "DependentServices":  [

                          ],
    "MachineName":  ".",
    "ServiceName":  "ssh-agent",
    "ServicesDependedOn":  [

                           ],
    "ServiceHandle":  {
                          "IsInvalid":  false,
                          "IsClosed":  false
                      },
    "Status":  1,
    "ServiceType":  16,
    "StartType":  4,
    "Site":  null,
    "Container":  null,
    "Name":  "ssh-agent",
    "RequiredServices":  [

                         ]
}

You name.json file doesn't have correct syntax. Can you post name.json content? — mk23, Jul 23 '21 at 11:48
That's probably the UTF-16 BOM that PowerShell commands tend to add. — Álvaro González, Jul 23 '21 at 11:59
@ÁlvaroGonzález How can i resolve this? Just read the file as utf-16? — Florian Taut, Jul 23 '21 at 12:02

score 3 · Accepted Answer · answered Jul 23 '21 at 12:11

3

Your JSON file seems to have a different encoding, maybe is utf16le instead of utf8.

I replicated your scenario and found help here: Strange unicode characters when reading in file in node.js app

fs.readFile(file, 'utf16le', (err, data)...

answered Jul 23 '21 at 12:11

Leo

1,990
1
13
20

Thank you @Leo. This at least removes the crazy icons at the beginning of the line. But still i receive the following error when trying `JSON.parse(data)`: ```undefined:1 { ^ SyntaxError: Unexpected token in JSON at position 0``` – Florian Taut Jul 23 '21 at 12:15
As i just found out opening the file with notepad++ the encoding is also not utf16le its: UCS-2-LE-BOM – Florian Taut Jul 23 '21 at 12:19
@FlorianTaut did you try to use iconv? https://stackoverflow.com/a/14404102/1724128 – Leo Jul 23 '21 at 12:22
I'm reading the answer above and found that "Unfortunately node.js only supports UTF-16 Little Endian or UTF-16LE", maybe there is a way to output powershell json in utf-8 – Leo Jul 23 '21 at 12:23
1

Using `iconv` i could now finally solve the issue by converting it from `UTF18-LE` to `UTF-8`. Thank you Leo! – Florian Taut Jul 23 '21 at 12:53

score 0 · Answer 2 · answered Jul 23 '21 at 11:58

0

I think a problem occurred because of line breakers and tabs. I tried to parse your code and it was parsed successfully. Please, try to make your JSON data to be one line. Like here: How to remove all line breaks from a string.

answered Jul 23 '21 at 11:58

Raximjon Komiljonov

76
4

I already did that. PowerShell's `-Compress` parameter returns the json object as a one liner. But this doens't resolve the issue. – Florian Taut Jul 23 '21 at 12:02
Try to print JSON data to console (console.log) your JSON data, I think it is different than you expected. My test result: https://prnt.sc/1er9fvh – Raximjon Komiljonov Jul 23 '21 at 12:05
Komiljinov As mentioned, i tried it doenst work. This it whats printed when use the `-Compress` parameter to receive a minified one liner: `��{"CanPauseAndContinue":false,"CanShutdown":false,"CanStop":false,"DisplayName":"OpenSSH Authentication Agent","DependentServices":[],"MachineName":".","ServiceName":"ssh-agent","ServicesDependedOn":[],"ServiceHandle":{"IsInvalid":false,"IsClosed":false},"Status":1,"ServiceType":16,"StartType":4,"Site":null,"Container":null,"Name":"ssh-agent","RequiredServices":[]}` – Florian Taut Jul 23 '21 at 12:11

score 0 · Answer 3 · answered Jul 23 '21 at 12:21

Maybe its the same problem as when you're writing PHP scripts that use session objects.

In PHP, when a file is encoded with UTF8 with BOM, the PHP script that uses session completely break. To solve this, I generally open the file in Notepad++, goes to Format -> UTF8 (without BOM), and then save it again. Always works for me. This might be your case here, since these broken characters appears to be on the beginning of your file, that's exacly where BOM is.

This answer might clarify about UTF8 and BOM.

SridharKritha · Answer 4 · 2021-10-28T17:22:03.443

0

Normally the error message having some strange characters (like �) gives hint that there maybe some problem in the encoding.

A few nodejs readFile api's with different encoding

fs.readFile(file, 'utf8', function(err, data) { ... });

fs.readFile(file, 'utf16le', function(err, data) { ... }); // le - little endian

fs.readFile(file, 'ucs2', function(err, data) { ... });   // kind  of 'utf16le'

edited Oct 28 '21 at 17:22

answered Oct 28 '21 at 17:16

SridharKritha

8,481
2
52
43

JSON.parse(): SyntaxError: Unexpected token � in JSON at position 0

4 Answers4