WebRequest to connect to the Wikipedia API

Question

This may be a pathetically simple problem, but I cannot seem to format the post webrequest/response to get data from the Wikipedia API. I have posted my code below if anyone can help me see my problem.

    string pgTitle = txtPageTitle.Text;

    Uri address = new Uri("http://en.wikipedia.org/w/api.php");

    HttpWebRequest request = WebRequest.Create(address) as HttpWebRequest;

    request.Method = "POST";
    request.ContentType = "application/x-www-form-urlencoded";

    string action = "query";
    string query = pgTitle;

    StringBuilder data = new StringBuilder();
    data.Append("action=" + HttpUtility.UrlEncode(action));
    data.Append("&query=" + HttpUtility.UrlEncode(query));

    byte[] byteData = UTF8Encoding.UTF8.GetBytes(data.ToString());

    request.ContentLength = byteData.Length;

    using (Stream postStream = request.GetRequestStream())
    {
        postStream.Write(byteData, 0, byteData.Length);
    }

    using (HttpWebResponse response = request.GetResponse() as HttpWebResponse)
    {
        // Get the response stream.
        StreamReader reader = new StreamReader(response.GetResponseStream());

        divWikiData.InnerText = reader.ReadToEnd();
    }

At first glance you're code looks good. How does the problem specifically present itself? What's the exception? — Tommi Forsström, Apr 21 '09 at 15:06
The exception is : The remote server returned an error: (417) Expectation failed. — NickJ, Apr 21 '09 at 15:28
You need to add System.Net.ServicePointManager.Expect100Continue = false; See http://stackoverflow.com/questions/566437/http-post-returns-the-error-417-expectation-failed-c-resolved — Keltex, Apr 21 '09 at 15:41
I would like to give you both some rep points for being so helpful. But I don't have enough yet. Anyway thanks guys. I will be visiting the Stack again. Keltex- maybe you could just edit your post with the answer so that other people can see the answer. cheers Nick — NickJ, Apr 21 '09 at 16:01

score 7 · Accepted Answer · edited May 23 '17 at 12:25

7

You might want to try a GET request first because it's a little simpler (you will only need to POST for wikipedia login). For example, try to simulate this request:

http://en.wikipedia.org/w/api.php?action=query&prop=images&titles=Main%20Page

Here's the code:

HttpWebRequest myRequest =
  (HttpWebRequest)WebRequest.Create("http://en.wikipedia.org/w/api.php?action=query&prop=images&titles=Main%20Page");
using (HttpWebResponse response = (HttpWebResponse)myRequest.GetResponse())
{
    string ResponseText;
    using (StreamReader reader = new StreamReader(response.GetResponseStream()))
    {
        ResponseText = reader.ReadToEnd();
    }
}

Edit: The other problem he was experiencing on the POST request was, The exception is : The remote server returned an error: (417) Expectation failed. It can be solved by setting:

System.Net.ServicePointManager.Expect100Continue = false;

(This is from: HTTP POST Returns Error: 417 "Expectation Failed.")

edited May 23 '17 at 12:25

Community

1
1

answered Apr 21 '09 at 15:11

Keltex

26,220
11
79
111

Thanks Keltex, I can get the Get working easily enough. using: WebRequest req = WebRequest.Create(address +"?" + data) as WebRequest; using (WebResponse resp = req.GetResponse() as WebResponse) { StreamReader readme = new StreamReader(resp.GetResponseStream()); divWikiData.InnerText = readme.ReadToEnd(); } This Post request still defats me - and I need Post for some of the actions the api allows.. – NickJ Apr 21 '09 at 15:26
Nick, can you please let us know what kind of problem your POST request is displaying. What's the exception you get? How is it behaving? – Tommi Forsström Apr 21 '09 at 15:28
The exception is : The exception is : The remote server returned an error: (417) Expectation failed. – NickJ Apr 21 '09 at 15:32
Keltex thanks again , for the Get request code. I really need some idea of how to use the Get request, as eventually I would like to try login and edit pages. – NickJ Apr 21 '09 at 15:36
1

You need to add System.Net.ServicePointManager.Expect100Continue = false; See http://stackoverflow.com/questions/566437/http-post-returns-the-error-417-expectation-failed-c-resolved – Keltex Apr 21 '09 at 15:40
I found that you *have* to set the `HttpWebRequest.UserAgent` property [\[^\]](http://www.mediawiki.org/wiki/API:Quick_start_guide#Identifying_your_client), and identify your client when calling the API. Else you will get a 403 Error back. So, I would add `myRequest.UserAgent = "MyWikiClient\1.0";` or the like, to Keltex's code. – lazo Apr 16 '11 at 12:03

score 1 · Answer 2 · answered Apr 22 '10 at 14:20

1

I'm currently in the final stages of implementing an C# MediaWiki API which allows the easy scripting of most MediaWiki viewing and editing actions.

The main API is here: http://o2platform.googlecode.com/svn/trunk/O2%20-%20All%20Active%20Projects/O2_XRules_Database/_Rules/APIs/OwaspAPI.cs and here is an example of the API in use:

var wiki = new O2MediaWikiAPI("http://www.o2platform.com/api.php");

wiki.login(userName, password);

var page = "Test"; // "Main_Page";

wiki.editPage(page,"Test content2");

var rawWikiText = wiki.raw(page);
var htmlText = wiki.html(page);

return rawWikiText.line().line() + htmlText;

answered Apr 22 '10 at 14:20

Dinis Cruz

4,161
2
31
49

1

Sorry about that, I moved that API recently to a more central location. You can find that file here: http://o2platform.googlecode.com/svn/trunk/O2_Scripts/APIs/MediaWiki/OwaspWikiAPI.cs this is the main API used on that file http://o2platform.googlecode.com/svn/trunk/O2_Scripts/APIs/MediaWiki/O2MediaWikiAPI.cs and this is a GUI tool built on top of those APIS http://o2platform.googlecode.com/svn/trunk/O2_Scripts/Tools/MediaWikiEditor.cs.o2 If you want to try those scripts, they are part of the O2 Platform which you can get from http://o2platform.com/ – Dinis Cruz Jun 15 '10 at 12:07

score 0 · Answer 3 · answered Apr 21 '09 at 15:10

You seem to be pushing the input data on HTTP POST, but it seems you should use HTTP GET.

From the MediaWiki API docs:

The API takes its input through parameters in the query string. Every module (and every action=query submodule) has its own set of parameters, which is listed in the documentation and in action=help, and can be retrieved through action=paraminfo. http://www.mediawiki.org/wiki/API:Data_formats

WebRequest to connect to the Wikipedia API

3 Answers3

Linked