Running Apache DefaultHttpClient and redeploying web app in Tomcat means Permgen space

Question

I have been around this problem few months and now I am sure HttpClient is the responsable of my permament PermGen Space errors.

My web app uses a Servlet. One of the functionalities allows users to connect to remote devices and request via HTTP status of this devices using REST services.

The problem arises when I redeploy Tomcat (in development time, I am not in production) and PermGen appears. I know the problem is related to this HttpClient because of the trace:

jun 19, 2013 4:51:34 PM org.apache.catalina.core.StandardWrapperValve invoke
SEVERE: El Servlet.service() para el servlet [dispatcher] en el contexto con ruta [] lanzó la excepción [Handler processing failed; nested exception is java.lang.OutOfMemoryError: PermGen space] con causa raíz
java.lang.OutOfMemoryError: PermGen space
    at java.lang.ClassLoader.defineClass1(Native Method)
    at java.lang.ClassLoader.defineClass(ClassLoader.java:791)
    at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
    at org.apache.catalina.loader.WebappClassLoader.findClassInternal(WebappClassLoader.java:2904)
    at org.apache.catalina.loader.WebappClassLoader.findClass(WebappClassLoader.java:1173)
    at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1681)
    at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1559)
    at org.apache.http.impl.cookie.RFC2109Spec.<init>(RFC2109Spec.java:83)
    at org.apache.http.impl.cookie.RFC2965Spec.<init>(RFC2965Spec.java:67)
    at org.apache.http.impl.cookie.BestMatchSpec.getStrict(BestMatchSpec.java:75)
    at org.apache.http.impl.cookie.BestMatchSpec.getVersion(BestMatchSpec.java:209)
    at org.apache.http.client.protocol.RequestAddCookies.process(RequestAddCookies.java:202)
    at org.apache.http.protocol.ImmutableHttpProcessor.process(ImmutableHttpProcessor.java:109)
    at org.apache.http.protocol.HttpRequestExecutor.preProcess(HttpRequestExecutor.java:176)
    at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:519)
    at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
    at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
    at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784)
    at com.pe.f.web.PlantController.getStatus(PlantController.java:431)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:601)
    at org.springframework.web.method.support.InvocableHandlerMethod.invoke(InvocableHandlerMethod.java:219)
    at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:132)
    at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:100)
    at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:604)
    at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:565)
    at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:80)
    at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:923)
    at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:852)
    at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:882)

Where

at com.pe.f.web.PlantController.getStatus(PlantController.java:431)

Is the method that launches the HTTP request.

I have two main questions I need to solve:

1- Is this problem specificly related to HttpClient or am I doing something wrong?

Here is my code:

for(Gdu g : p.getGduCollection()){

    try {
        DefaultHttpClient httpClient = new DefaultHttpClient();
        httpClient.getParams().setParameter(CoreConnectionPNames.CONNECTION_TIMEOUT, 30000);
        get = new HttpGet("http://" 
                        + p.getInstallationId().getIprouter() 
                        + ":" 
                        + g.getHttpPort().toString()
                        + "/FSP/status");
        HttpResponse response = httpClient.execute(get);

        if (response.getStatusLine().getStatusCode() != 200) {
            throw new RuntimeException("Failed : HTTP error code : "
                           + response.getStatusLine().getStatusCode());
        }
        StringBuilder inputStringBuilder = new StringBuilder();
        BufferedReader br = new BufferedReader(
                         new InputStreamReader((response.getEntity().getContent())));
        String line;

        while ((line = br.readLine()) != null) {
            if(!line.contains("<?xml")){
                inputStringBuilder.append(line);
            }                    
        }

        httpClient.getConnectionManager().shutdown();
        logger.debug(inputStringBuilder.toString());
        Document xml = loadXMLFromString(inputStringBuilder.toString());

        // sets the date
        status.setDate(new SimpleDateFormat("dd/MM/yyyy HH:mm:ss").parse(xml.getElementsByTagName("time").item(0).getTextContent()));

        // add power of each GDU
        Float f = Float.parseFloat(status.getPower().split(" ")[0]);
        f += Float.parseFloat(xml.getElementsByTagName("power").item(0).getTextContent().split(" ")[0]);
        status.setPower(f.toString() + " kW");

        // add the energy of each GDU
        Float f2 = Float.parseFloat(status.getEnergy().split(" ")[0].replace(",", "."));
        f2 += Float.parseFloat(xml.getElementsByTagName("energy").item(0).getTextContent().split(" ")[0]);
        status.setEnergy(String.format("%.3f", f2 - 0.005).replace(",", ".") + " MWh");

        NodeList nl = xml.getElementsByTagName("status");
        stat = 0;
        for(int j=0; j < nl.getLength(); j++){
            if(nl.item(j).getNodeType() == Node.ELEMENT_NODE){ // Node
                Element e = (Element) nl.item(j);
                stat += Integer.valueOf(e.getElementsByTagName("value").item(0).getTextContent());
            }
        }

        status.setPlantStatus(String.valueOf(stat + Integer.parseInt(status.getPlantStatus())));

        } catch (ConnectTimeoutException cte ) {
            status.setConnError("Unnable to communicate with plant. Reason: Connection Timeout.");
        } catch (SocketTimeoutException ste) {
            status.setConnError("Unnable to communicate with plant. Reason: Socket Timeout.");
        } catch (IOException ex) {
            status.setConnError("Unnable to communicate with plant. Reason: I/O Exception.");

        } finally {
            if(get != null){
                get.releaseConnection();
            }                
        }        
    }

My code opens a HTTP connection, requests the status and receives XML as a String. Once I get the XML I create an Status object with this information.

2- Do you suggest any other thinner HTTP client? HttpURLConnection maybe?

I know most of you will suggest to increase the Permament memory heap. I have read a lot on this topic. I don't see the point to increase the heap size beacuse in the end, if the problem should appear it will appear.

Thanks in advance

Update: Following SO post I have avoided the automatic retry request in HttpClient. I will keep performing tests in development server and production server before accept any answer.

References:

score 0 · Answer 1 · answered Jun 19 '13 at 20:09

0

I doubt that the problem is with HttpClient. Application redeploys tend to to eat up perm gen space because classes are being reloaded (and placed into perm gen space) upon every redeploy.

If the problem only appears during development due to application redeploys, then increasing perm gen size is the way to go. You can also try out e.g. JRebel to avoid full redeploys. Also restarting Tomcat every now and then helps.

answered Jun 19 '13 at 20:09

Jukka

4,583
18
14

Does that mean if I don't redeploy the application in production this problem should not appear? – Spacemonkey Jun 20 '13 at 07:47
Probably. But you need to test it. Eventually you will need to redeploy in production as well. Personally, I handle production redeploys (updates) by running multiple Tomcat nodes behind an LB, which allows me to restart Tomcat (thus avoiding perm gen issues). – Jukka Jun 20 '13 at 07:52
Then I should test it in production. You don't really think this is related to HttpClient? Why then the trace? – Spacemonkey Jun 20 '13 at 08:36
Like I said, I doubt it. Stacktraces always point to code you are running when the exception is thrown. Your code just happens to use HttpClient, hence it is visible on the stacktrace. – Jukka Jun 20 '13 at 08:47

score 0 · Answer 2 · answered Jun 27 '13 at 13:33

0

You can allow your development tomcat to unload all the perm gen classes with this command line parameter:

 -XX:+CMSClassUnloadingEnabled

answered Jun 27 '13 at 13:33

Will

6,179
4
31
49

I will check it, but in order to follow your advice I would follow Digulla's advice in this post http://stackoverflow.com/q/3334911/1242531 on `UseConcMarkSweepGC` – Spacemonkey Jun 27 '13 at 14:28

Running Apache DefaultHttpClient and redeploying web app in Tomcat means Permgen space

2 Answers2