0

I'm working with a public website that uses javascript and url-hiding, so I cannot tell if it's php/asp, nor how I can set up the POST. Here's the website... (https://emops.twse.com.tw/server-java/t58query)

What I need to do is programmatically click on each of the VIEW button and retrieve the html/response in the window that pops up, and close the window after it's loaded into html/response/string

Normally I'd do this from database with sql command or export it into Excel, but I don't have access for this website. I am not very familiar with the front-end way of maybe using javascripts to do this...

The only "method" I can think up is to use WebBrowser Control to load the page and invoke "onclick" action. But after that, I'm not sure if I can use shell32.dll to FindWindow and to read all of its content then close it... and I prefer not to involve unmanaged code.

I've only managed to parse the page with HtmlAgilityPack, but I haven't found how to invoke the CLICK (act as a POST) and how to get response from the pop'd up window (from CLICKING)...

private void ParseHtmlUsingAgilityPack(WebBrowser wbMOP)
    {
        HTMLDocument htmlDoc = (HTMLDocument)wbMOP.Document;

        HtmlDocument doc = new HtmlDocument();
        doc.LoadHtml(htmlDoc.documentElement.innerHTML);
        HtmlNodeCollection bodyNodes = doc.DocumentNode.SelectNodes("//form//table//tbody//td//input/@onclick"); //input/@value

        List<string> listOfAttStr = new List<string>();

        if (bodyNodes != null)
        {
            foreach (HtmlNode link in bodyNodes)
            {
                foreach (HtmlAttribute ha in link.Attributes)
                {
                    if (ha.ValueLength > 24 && ha.Value.Substring(0, 22) == "document.fm_t05sr01_1.")
                    {
                        listOfAttStr.Add(ha.Value);
                    }
                }

            }

            // write into text file here...
        }

    }
Danny293891
  • 3
  • 2

1 Answers1

0

I would use selenium for this kind of tasks. Look for example at this answer: https://stackoverflow.com/a/14943813/11776368 for handling popup windows.

rmac38
  • 123
  • 7
  • Thanks for feedback, but I am having no problem getting the main window or the pop up window process handle by using C++/WinAPI's functions. The problem is even with those handle, you can't look into innerHTML of a process window without some sort of "managing" which I was hoping to get here. Also, with FindWindow, with sleep or wait, it does grab it, but it doesn't detect whether the page has been fully loaded. Therefore, I was hoping to use WebBrowser to trigger LoadComplete event and get the document – Danny293891 Jul 18 '19 at 23:55
  • You can do this with selenium. After you detected the popup window you just have to wait for example until the Table in the popup is visible. (wait.until visibilityOfElementLocated ) – rmac38 Jul 19 '19 at 04:26