2

I need to parse HTML elements from a webpage in Java, fill out a form and submit it to simulate a login.

I've been doing a little bit of research and here's what I did using the jsoup library.

However, after implementing this, I realized jsoup is only for parsing and can't perform button click, for example.

In this example, I need to perform a click on submitButton element.

try {
    Document htmlDoc = Jsoup.connect(MY_URL).get();
    Element loginForm = htmlDoc.getElementById("login-form");
    Element usernameField = loginForm.getElementById("username");
    Element passwordField = loginForm.getElementById("passwd");
    Element submitButton = loginForm.getElementById("submit");

    usernameField.val(username);
    passwordField.val(password);

    // I need to simulate `submitButton` click so I can login

} catch (Exception e) {
    Log.e("TAG", e.toString());
}

Am I correct in my understanding that jsoup is not capable of doing this?

If so, is there a library for Android that can?

I understand there are many Java libraries but most can't work with Android (ex. HtmlUtil) due to javax limitation.

ᴛʜᴇᴘᴀᴛᴇʟ
  • 4,466
  • 5
  • 39
  • 73
  • Do you want to emulate a browser or do you just want to post a form? Why would you need to parse the form at all? After all, you are extracting no information from the form in the code snippet you supplied. – Lupinity Labs Nov 19 '16 at 03:06
  • I don't need to extract any info. All I need to do is verify if the provided username and password are valid. If I can simulate a click, i can verify if the provided id and password are valid or not. I know there are better methods to do this however I need to implement it this way temporarily – ᴛʜᴇᴘᴀᴛᴇʟ Nov 19 '16 at 03:10
  • The approach heavily depends on the nature of the site you want to probe. If it returns an error (404, 403, 401 for example), you can just post a username and password and look at the response. See https://developer.android.com/reference/java/net/HttpURLConnection.html for getting on the right track. If you search for HttpURLConnection and HTTP POST, you will get plenty of tutorials on how to POST form data to a URL. – Lupinity Labs Nov 19 '16 at 03:14
  • You need to find the target link and parameters that needs to be passed. You don't need to parse the page. – Gurwinder Singh Nov 19 '16 at 04:24
  • I already have the target link and parameters. Based on the the answer given below (link), I tried however I am getting the following response: "The most recent request was denied because it contained an invalid security token. Please refresh the page and try again." when I do the post method – ᴛʜᴇᴘᴀᴛᴇʟ Nov 19 '16 at 04:36
  • Also keep in mind: jsoup doesn't provide a JavaScript engine. Since you are on android I would recommend: use a WebView to call and render a website, the use JavaScript to modify/insert values/content and trigger the submission. – Frederic Klein Nov 19 '16 at 10:29
  • @FredericKlein so you are saying this is possible through `WebView` and `JavascriptInterface`? Do you have an example or a link I can look at? I know that we can call Java methods from HTML/Javascript if I modify the site however I am not looking for that solution. I want to perform a `Submit` button click on the site from Java – ᴛʜᴇᴘᴀᴛᴇʟ Nov 19 '16 at 14:23
  • This looks similar: http://stackoverflow.com/questions/29905821/android-webview-compile-a-form-and-submit-with-javascript. Just test the functionality in you desktop browser's JavaScript console, then use the JavaScript code in your android WebView and pass the result from the form submission to a JavaScriptInterface, so you can react in your Java code. – Frederic Klein Nov 19 '16 at 14:34
  • This might work. I will test it out. Thank you! – ᴛʜᴇᴘᴀᴛᴇʟ Nov 19 '16 at 15:06

1 Answers1

1

You can view this link. Provides a good answer.

The steps should be the following:

  1. Get to the login page
  2. Parse the html using jsoup for username and password
  3. Replace it with the actual username and password
  4. Use an http POST to post the username and password form.
Pritam Banerjee
  • 17,953
  • 10
  • 93
  • 108