Downloading a file from Selenium and chromedriver

Question

I can't get Selenium and Chrome (Canary) to download a file. I'm using Java and and Chrome 59/60 (because my tests are both for Windows and Linux) and I'm trying to start the download of a file from a webpage.

When I, from selenium, do NOT set the headless mode, the chrome window opens and the file is downloaded.

When I do set the --headless flag, the chrome window does not open and the download does not start.

    public static void chromeDownload() throws IOException, InterruptedException{
            
            ChromeOptions options = new ChromeOptions();
            String downloadFilepath = "";
            
            if (ValidateOS.isWindows()){
                System.out.println("This is a Windows system.");
                System.setProperty("webdriver.chrome.driver", "resources\\driver\\chromedriver.exe");
                options.setBinary("C:\\Users\\Juri\\AppData\\Local\\Google\\Chrome SxS\\Application\\chrome.exe");
                downloadFilepath = "C:\\";
            } else if (ValidateOS.isUnix()){
                System.out.println("This is a Unix system.");
                System.setProperty("webdriver.chrome.driver", "resources/driver/chromedriver");
                options.setBinary("/usr/bin/google-chrome");
                downloadFilepath = "/home/juri/";
            }
            
            // Manage the download
            HashMap<String, Object> chromePrefs = new HashMap<>();
            chromePrefs.put("profile.default_content_settings.popups", 0);
            chromePrefs.put("download.default_directory", downloadFilepath);
    
            // Save Chrome Options
            HashMap<String, Object> chromeOptionsMap = new HashMap<>();
            options.setExperimentalOption("prefs", chromePrefs);
            options.addArguments("--headless --disable-gpu");
            
            DesiredCapabilities cap = DesiredCapabilities.chrome();
            cap.setCapability(ChromeOptions.CAPABILITY, chromeOptionsMap);
            cap.setCapability(CapabilityType.ACCEPT_SSL_CERTS, true);
            cap.setCapability(ChromeOptions.CAPABILITY, options);
            
            ChromeDriver driver = new ChromeDriver(cap);
                    
            driver.get("http://localhost/my-test-page.html");

            driver.findElement(By.id("download")).click(); 
            Thread.sleep(5000); // wait 5 seconds for a small file to download.. yes.. I know...
            driver.quit();
        }

At the Click, in GUI mode the download starts. In Headless mode, it doesn't.

How to solve?

OT

I am using Chrome Canary which at its v.60 comes with the --headless feature. Ultra handy for running the grabber on a server without gui. But, for the same reason.. I find it useless to download Chrome on a server without GUI. Beside the main question.. I wonder if you, developers, think that it is okay to install chrome on a Linux server just for starting it in headless mode.

Update: I'm still looking for a solution if someone will ever read this :/ Search results there are a few and I tried them all

Answer 1

Solved by adapting the code found at this link: Download files in Java, Selenium using ChromeDriver and headless mode

For those who wonder how's my code now...

public static void chromeDownload(String address, String Headless, String DownDir) throws IOException, InterruptedException{

    ChromeOptions options = new ChromeOptions();
    String downloadFilepath = DownDir;

    if (ValidateOS.isWindows()){
        System.out.println("This is a Windows system.");
        System.setProperty("webdriver.chrome.driver", "resources\\driver\\chromedriver.exe");
        //options.setBinary("C:\\Users\\Juri\\AppData\\Local\\Google\\Chrome SxS\\Application\\chrome.exe");
        // If this is commented, the grabber will use the main Chrome
    } else if (ValidateOS.isUnix()){
        System.out.println("This is a Unix system.");
        System.setProperty("webdriver.chrome.driver", "resources/driver/chromedriver");
        options.setBinary("/usr/bin/google-chrome");
    }

    switch (Headless.toUpperCase()){
        case "TRUE":
            options.addArguments("--headless --disable-gpu");
            break;
        case "FALSE":
        default:
            options.addArguments("--window-size=1152,768");
            break;
    }
    options.addArguments("--test-type");
    options.addArguments("--disable-extension");

    ChromeDriverService driverService = ChromeDriverService.createDefaultService();
    ChromeDriver driver = new ChromeDriver(driverService, options);

    Map<String, Object> commandParams = new HashMap<>();
    commandParams.put("cmd", "Page.setDownloadBehavior");
    Map<String, String> params = new HashMap<>();
    params.put("behavior", "allow");
    params.put("downloadPath", downloadFilepath);
    params.put("cmd", "Page.setDownloadBehavior");

    commandParams.put("params", params);
    ObjectMapper objectMapper = new ObjectMapper();
    CloseableHttpClient httpClient = HttpClientBuilder.create().build();
    String command = objectMapper.writeValueAsString(commandParams);
    String u = driverService.getUrl().toString() + "/session/" + driver.getSessionId() + "/chromium/send_command";
    HttpPost request = new HttpPost(u);
    request.addHeader("content-type", "application/json");
    request.setEntity(new StringEntity(command));
    httpClient.execute(request);

    driver.get(address);

    driver.findElement(By.id("download")).click(); 
    driver.quit();
}

Answer 2

Do you anticipate using "Docker"? launch dockerized Ubuntu with selenium Grid and any amount of browsers. Or just browser without Selenium grid.

You'd have no need to use headless mode on the other hand you could you multi-threading. Ex:

1. Download the file.

2. Then just launch in using the following command: docker-compose up -d

3. a few tweaks per your server to access grid server on "localhost"
   http://localhost:4444/grid/console http://localhost:4444/wd/hub

4. Use the following code:

   WebDriver driver = new RemoteWebDriver(new URL("http://localhost:4444/wd/hub")); //handle exception on URL() constructor.

5. you have lightweight virtual machines with browsers any amount which your sever can handle. And no need of --headless mode

It's not the solution of the initial issue though.