0

I'm trying to import a file to Google Big Query. When I import it using a BigqueryJob, I get an error:

 Error detected while parsing row starting at position: 0. Error: Bad character (ASCII 0) encountered.

I've solved this by replacing the ASCII 0 character in notepad ++ or with PowerShell with the following Script:

   $configFiles = Get-ChildItem -Path C:\InputPath\* 
   foreach ($file in $configFiles)
   {
         (Get-Content $file.PSPath) |
         Foreach-Object { $_ -replace "`0", "" } |
         Set-Content $file.PSPath
   }

But I need to automate this, so I'm using Google Cloud DataFusion, but when I open this file with the wrapper, I get a screen with a Square symbol (Couldn't copy the character, so I pasted an image): ASCII 0 in DataFusion

What can I do to load this file with DataFusion?

If I open this same file in notepad/notepad++ I can see the characters like any other txt file. Thanks!

Cristian Avendaño
  • 427
  • 3
  • 17
  • 1
    It is not possible to insert a custom script in data fusion. A work around for this is to create a [custom java plugin](https://stackoverflow.com/questions/60188527/is-it-possible-to-add-my-own-custom-transformation-plugin-to-cloud-data-fusion-e) that will remove the ["Bad Characters"](https://stackoverflow.com/questions/9075603/convert-utf-16-unicode-characters-to-utf-8-in-java) and use that plugin in data fusion. – Ricco D Dec 11 '20 at 07:48
  • 1
    The bad characters encountered are when importing in BigQuery are most likely [utf-16 characters](https://stackoverflow.com/questions/11632859/bad-character-ascii-0-encountered-while-importing-data). I will file a feature request for this. – Ricco D Dec 11 '20 at 07:49
  • Feel free to add on the [public issue tracker](https://issuetracker.google.com/175365013) – Ricco D Dec 11 '20 at 07:59

0 Answers0