Send to github via curl command line (Windows)

Question

I asked a related question and realized I wasn't asking the right question (i.e., this isn't about git).

The question is how to push a project to github without first creating the project in the clouds using R. Currently you can do this from the git command line in RStudio using info from this question.

Now I'm trying to make that into R code now from a Windows machine (Linux was easy). I'm stuck at the first step using curl from the command line via an R system call. I'll show what I have and then the error message (Thanks to SimonO101 for getting me this far.). Per his comments below I've edited heavily to reflect the problem as it:

R Code:

repo <- "New"
user <- "trinker"
password <- "password"

url <- "http://curl.askapache.com/download/curl-7.23.1-win64-ssl-sspi.zip"
tmp <- tempfile( fileext = ".zip" )
download.file(url,tmp)
unzip(tmp, exdir = tempdir())

system(paste0(tempdir(), "/curl http://curl.haxx.se/ca/cacert.pem -o " , 
    tempdir() , "/curl-ca-bundle.crt"))

cmd1 <- paste0(tempdir(), "/curl -u '", user, ":", password, 
    "' https://api.github.com/user/repos -d '{\"name\":\"", repo, "\"}'")

system(cmd1)

cmd2 <- paste0(tempdir(), "/curl -k -u '", user, ":", password, 
    "' https://api.github.com/user/repos -d '{\"name\":\"", repo, "\"}'")

system(cmd2)

Error Messages (same for both approaches):

> system(cmd1)
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100    12    0     0  100    12      0     24 --:--:-- --:--:-- --:--:--    30
100    47  100    35  100    12     65     22 --:--:-- --:--:-- --:--:--    83{
  "message": "Bad credentials"
}

I know all the files are there because:

> dir(tempdir())
[1] "curl-ca-bundle.crt"   "curl.exe"             "file1aec62fa980.zip"  "file1aec758c1415.zip"

It can't be my password or user name because this works on Linux Mint (the only difference is the part before curl):

repo <- "New"
user <- "trinker"
password <- "password"


cmd1 <- paste0("curl -u '", user, ":", password, 
    "' https://api.github.com/user/repos -d '{\"name\":\"", repo, "\"}'")

system(cmd1)

NOTE: Windows 7 machine. R 2.14.1

This does, as you suggest just work on my Mac as is. Did you try the `-k` option as the error suggests? i.e. `cmd1 <- paste0("curl -k -u '", user, ":", password, "' https://api.github.com/user/repos -d '{\"name\":\"", repo, "\"}'")` — Simon O'Hanlon, Feb 23 '13 at 16:17
Yes and it seems to hang up indefinitely. I'll try it again. — Tyler Rinker, Feb 23 '13 at 16:18
Ah, it did hang for me too, but then I accidentally closed my MBP lid. When I reopened it, the command had completed and the repo had appeared on GitHub...! Go figure. — Simon O'Hanlon, Feb 23 '13 at 16:19
And in fact, since the first time when it hung, it now clones a new repo to GitHub straightaway, no problems without hanging — Simon O'Hanlon, Feb 23 '13 at 16:21
@SimonO101 I added the message I get from doing `-k` that now. — Tyler Rinker, Feb 23 '13 at 16:26
It sounds too obvious to be the case, but that looks like either your username or password is wrong?! Also have you checked GitHub and you aren't trying to overwrite the name of an already existing repo? — Simon O'Hanlon, Feb 23 '13 at 16:32
@SimonO101 No this is not the case. I just tried this on linux Mint (so easy it makes me wanna cry) and it works with ease. same code pasted from above. — Tyler Rinker, Feb 23 '13 at 16:42
@trinker Did the below not work for you? I am interested to see this working! — Simon O'Hanlon, Feb 25 '13 at 08:40
@SimonO101 I thought I left a comment but I don't see it. No it did not work. It got me closer though. I reposted above what's going on. I suspect it has to do with the https vs. http but am not sure. — Tyler Rinker, Feb 25 '13 at 12:33
@trinker - Have you tried starting R with the `--internet2` option? In case you aren't sure you can right-click the executable or shortcut and select properties. --internet2 would go after the target (not enclosed in the quotes). — Simon O'Hanlon, Feb 25 '13 at 12:58
@SimonO101 I do that and get the same response :( This seems similar to [an issue](http://stackoverflow.com/questions/9305471/zip-file-error-in-reading-in-an-https-url) I had before but the error message isn't the same. Could this be done with RCurl instead. That may be an easier approach? — Tyler Rinker, Feb 25 '13 at 17:36
This is driving me nuts. It just works through `curl` on mac os x. I can't seem to construct the appropriate request through RCurl — Simon O'Hanlon, Feb 25 '13 at 21:29
Maybe try running the command escaping the `'` characters (incase there is some oddity on Windows?). So `system( paste0( tempdir() ,"/curl -u \'", user, ":", password, "\' https://api.github.com/user/repos -d \'{\"name\":\"", repo, "\"}\'") )` — Simon O'Hanlon, Feb 25 '13 at 21:51
Here's the near completed function that works well for Linux: https://github.com/trinker/reports/blob/master/R/repo2github.R — Tyler Rinker, Feb 26 '13 at 07:09
I think I have fixed it! See my updated answer below. We need to do some extra crazy escaping for windows cases and also it seems to matter if there is whitespace around the curly braces. — Simon O'Hanlon, Feb 26 '13 at 15:24

score 4 · Accepted Answer · edited May 23 '17 at 11:54

EDIT - After OP offered bounty

Ok, it turns out it is to do with some crazy windows character escaping on the command line. Essentially the problem was we were passing improperly formatted json requests to github.

You can use shQuote to properly format the offending portion of the curl request for Windows. We can test platform type to see if we need to include special formatting for Windows cases like so:

repo <- "NewRepository"
json <- paste0(" { \"name\":\"" , repo , "\" } ") #string we desire formatting
os <- .Platform$OS.type #check if we are on Windows
if( os == "windows" ){
json <- shQuote(json , type = "cmd" )
cmd1 <- paste0( tempdir() ,"/curl -i -u \"" , user , ":" , password , "\" https://api.github.com/user/repos -d " , json )
}

This worked on my Windows 7 box without any problems. I can update the GitHub script if you want?

OLD ANSWER

I did some digging around here and here and it might be that the answer to your problem is to update the curl-ca-bundle. It may help on Windows to get R to use the internet2.dll.

repo <- "New"
user <- "trinker"
password <- "password"

url <- "http://curl.askapache.com/download/curl-7.23.1-win64-ssl-sspi.zip"
tmp <- tempfile( fileext = ".zip" )
download.file(url,tmp)
unzip(tmp, exdir = tempdir())
system( paste0( "curl http://curl.haxx.se/ca/cacert.pem -o " , tempdir() , "/curl-ca-bundle.crt" ) )
system( paste0( tempdir(),"/curl", " -u \'USER:PASS\' https://api.github.com/user/repos -d \'{\"name\":\"REPO\"}\'") )

Again, I can't test this as I don't have access to my Windows box, but updating the certificate authority file seems to have helped a few other people. From the curl website, the Windows version of curl should look for the curl-ca-bundle.crt file in the following order:

application's directory
current working directory
Windows System directory (e.g. C:\windows\system32)
Windows Directory (e.g. C:\windows)
all directories along %PATH%

I'm attributing credit to you on this function. If you'd like me to use your actually name please email me via the github provided email address. Thanks again for your help. — Tyler Rinker, Feb 26 '13 at 18:25
@TylerRinker the bounty does not seem to have been awarded. I think this is because I originally posted this answer before the bounty was offered, and then edited it rather than posting a new answer. I m not sure whether you can manually award the bounty? Or will it just be awarded when the bounty period expires? I would hate for those lovely points to disappear into the SO ether... — Simon O'Hanlon, Feb 28 '13 at 15:14
Hmm I was thinking if I marked it correct you got the bounty. I think it went through. Let me know if not. We may have to wait until it expires. — Tyler Rinker, Feb 28 '13 at 17:27

Send to github via curl command line (Windows)

1 Answers1

Linked