0

I am using Linux machine for development and then deploying my script files to Azure function app which is a Windows machine.

Then I am copying shell script files to wasb from Azure function app.

I am getting following error when running shell script in Edge node on Azure HDI cluster

$'\r': command not found

My script files are not executed properly.

What is the best way to convert dos2unix option in C# Azure function?

Panagiotis Kanavos
  • 120,703
  • 13
  • 188
  • 236
Galet
  • 5,853
  • 21
  • 82
  • 148
  • Do you use Notepad++? When you create a file on Windows, you could select Unix type. – Shui shengbao Oct 25 '17 at 07:33
  • @Walter-MSFT I am not using Windows machine. I am copying files from github to Azure function and then to WASB. Files are stored in Azure function app which is a windows machine. I am pushing files from Linux machine to github. I need a automated approach not manual. – Galet Oct 25 '17 at 07:41
  • 1
    There's no such thing as `Linux File Format` and `Windows File Format`. The newline characters for *text files* are different on the two OS but C# works with both. The newline in Linux is `\n`. In Windows it's `\r\n`. Neither uses `\r` – Panagiotis Kanavos Oct 25 '17 at 08:42

5 Answers5

2

You don't have to do anything special to read a file that contains Unix-style newlines (\n) instead of \r\n. .NET IO methods treat both as newlines.

You could write 

var lines-File.ReadAllLines("myUnixText.txt");

or 

using(var reader=File.OpenText("myUnixText.txt"))
{
    string line;
    while( (line=reader.ReadLine()) !=null)
    {
        // Do something
    }
}

to read lines whether the line ending is \r or \n

To prove it :

var numbers = new[]{1,2,3,4,5,6};
var lines=String.Join("\n",numbers);
File.WriteAllText("myUnixText.txt",lines);

var newLines=File.ReadAllLines("myUnixText.txt");

Debug.Assert(newLines.Length==6);

Even though only a single string was written, File.ReadAllLines read 6 lines from the file

Panagiotis Kanavos
  • 120,703
  • 13
  • 188
  • 236
1

You could upload your file to Linux and change file format. 

dos2unix <filename>

Another easy solution is that if you install Notepad++, it supports automatic conversion format to Unix(LF).

enter image description here

Shui shengbao
  • 18,746
  • 3
  • 27
  • 45
1

I am calling Dos2Unix(filePath) method before copying file from Azure function to WASB.

Call method:-

Dos2Unix(D:\home\site\repository\sample.sh);

Following method actually works for me in C# Azure function.

private void Dos2Unix(string fileName)
{
    const byte CR = 0x0D;
    const byte LF = 0x0A;
    byte[] data = File.ReadAllBytes(fileName);
    using (FileStream fileStream = File.OpenWrite(fileName))
    {
        BinaryWriter bw = new BinaryWriter(fileStream);
        int position = 0;
        int index = 0;
        do
        {
            index = Array.IndexOf<byte>(data, CR, position);
            if ((index >= 0) && (data[index + 1] == LF))
            {
                // Write before the CR
                bw.Write(data, position, index - position);
                // from LF
                position = index + 1;
            }
        }
        while (index >= 0);
        bw.Write(data, position, data.Length - position);
        fileStream.SetLength(fileStream.Position);
    }
}

Update 1:-

Here is my code to upload files(*.sh, *.jar, *.img, etc) to blob storage.

public bool UploadBlobFile(string containerName, string blobName, string filePath)
{

    try{
        CloudBlobContainer container = blobClient.GetContainerReference(containerName);
        CloudBlockBlob blob = container.GetBlockBlobReference(blobName);
        // convert dos2unix format
        Dos2Unix(filePath);
        using (var fileStream = System.IO.File.OpenRead(filePath))
        {
            blob.UploadFromStream(fileStream);
        }
        return true;
    } catch (Exception e) {
        log.Info("Exception: " + e);
        return false;
    }
}
Galet
  • 5,853
  • 21
  • 82
  • 148
  • Instead of doing this, try a simple `File.OpenText(fileName)`. You'll see that the TextReader can handle `\n` just fine – Panagiotis Kanavos Oct 25 '17 at 08:44
  • Or write `File.ReadAllLines(fileName)` to get all lines as an array. You don't need any of this code – Panagiotis Kanavos Oct 25 '17 at 08:55
  • @PanagiotisKanavos What should I do after I use File.OpenText(fileName) or File.ReadAllLines(fileName)?. Because currently I am using this code to upload blob file. Dos2Unix(filePath); using (var fileStream = System.IO.File.OpenRead(filePath)) { blob.UploadFromStream(fileStream); } – Galet Oct 25 '17 at 09:41
  • you don't need the `Dos2Unix` method at all. You don't need to do anything. .NET can handle `\n` just fine, as I show in my answer. You *did* use the `c#` tag so I assume your script is a C# file. In fact, you should post your *actual* code, not the error message – Panagiotis Kanavos Oct 25 '17 at 12:45
  • @PanagiotisKanavos I have updated the code. Can you tell me what I should I modify the above code? – Galet Oct 25 '17 at 13:00
  • is *this* the code that throws the error?? Where? Update the *question* with the actual script that throws the error. As for this code, simply remove the call to `dos2Unix`. If the Azure Function script is written in C# it can understand files with `\`n just fine – Panagiotis Kanavos Oct 25 '17 at 13:15
0

It seems that you have Windows style line endings (\r\n) - you need to change them to unix style (\n).

Refer: '\r': command not found and Convert DOS line endings to Linux line endings in vim

CHEEKATLAPRADEEP
  • 12,191
  • 1
  • 19
  • 42
0

There is a tool in sourceforge that can do it. Is has been actualized recently a runs very well.

Josem
  • 378
  • 7
  • 15