how to get pdf file from GCS on app-engine - java

Question

I am running an app on google-app-engine.

trying to get txt from pdf-file on google-cloud-storage.

when I run my code locally it succeeds, but when running on appengine it fails with org.pdfbox.exceptions.WrappedIOException

here is my code:

import com.google.cloud.storage.*;
import org.pdfbox.pdfparser.PDFParser;
import org.pdfbox.pdmodel.PDDocument;
import org.pdfbox.util.PDFTextStripper;


public class Download {

    public static String perform(String bucket, String file) throws IOException {
        byte[] fileByte = download(bucket, file);
        String pdfFileTxt = pdf2txt(fileByte);
        return pdfFileTxt;

    }

    public static byte[] download(String bucketName, String fileId) throws IOException {
        Storage storage = StorageOptions.getDefaultInstance().getService();
        BlobId blobId = BlobId.of(bucketName, fileId);
        Blob blob = storage.get(blobId);
        return blob.getContent();
    }

    public static String pdf2txt(byte[] byteArr) throws IOException {
        InputStream stream = new ByteArrayInputStream(byteArr);
        PDFParser parser = new PDFParser(stream);
        parser.parse();
        PDDocument pdDoc = new PDDocument(parser.getDocument());
        return new PDFTextStripper().getText(pdDoc);
    }
}

the code fails on parser.parse(); with org.pdfbox.exceptions.WrappedIOException - no other message is added :(

the download from storage - actually succeeds. If I log the data I get something like:

%PDF-1.3
%����
7 0 obj
<</Linearized 1/L 7945/O 9/E 3524/N 1/T 7656/H [ 451 137]>>
endobj

13 0 obj
<</DecodeParms<</Columns 4/Predictor 12>>/Filter/FlateDecode/ID[<4DC91A1875A6D707AEC203BB021C93A0><F6C92B368A8A13408457A1D395A37EB9>]/Index[7 21]/Info 6 0 R/Length 52/Prev 7657/Root 8 0 R/Size 28/Type/XRef/W[1 2 1]>>stream
h�bbd``b`� ��H0�  6G ��#�4�,#��Ɲ_  L��
endstream
endobj
startxref
0
%%EOF
...  more ...

now is there anyway to overcome this? Maybe use different libraries? Since the code is running on appengine - it's quite difficult to track these errors.

Answer 1

PdfBox does not run on GAE. It uses not-allowed java classes.

as a work around you can download a modified pdfbx jar. thanks icyerasor's answer and using these instructions.

here are the full instructions:

download this folder now cd into your project dirctory and run the following commands:

mkdir local-maven-repo

.

mvn deploy:deploy-file -DgroupId=org.apache.pdfbox -DartifactId=pdfbox -Dversion=1.8.0-SNAPSHOT -Durl=file:./local-maven-repo/ -DrepositoryId=local-maven-repo -DupdateReleaseInfo=true -Dfile=/your/path/to/download/directory/pdfbox-GAE/pdfbox-1.8.0-SNAPSHOT.jar 

.

mvn deploy:deploy-file -DgroupId=org.apache.pdfbox -DartifactId=fontbox -Dversion=1.8.0-SNAPSHOT -Durl=file:./local-maven-repo/ -DrepositoryId=local-maven-repo -DupdateReleaseInfo=true -Dfile=/your/path/to/download/directory/pdfbox-GAE/dependencies/fontbox-1.8.0-SNAPSHOT.jar

in pom under project add :

<repositories>
    <repository>
        <id>local-maven-repo</id>
        <url>file:///${project.basedir}/local-maven-repo</url>
    </repository>
</repositories>

now edit your dependencies in pom:

      <dependency>
        <groupId>commons-logging</groupId>
        <artifactId>commons-logging</artifactId>
        <version>1.1.1</version>
    </dependency>
    <!-- https://mvnrepository.com/artifact/commons-logging/commons-logging-api -->
    <dependency>
        <groupId>commons-logging</groupId>
        <artifactId>commons-logging-api</artifactId>
        <version>1.1</version>
    </dependency>
    <!-- https://mvnrepository.com/artifact/commons-logging/commons-logging-adapters -->
    <dependency>
        <groupId>commons-logging</groupId>
        <artifactId>commons-logging-adapters</artifactId>
        <version>1.1</version>
    </dependency>
    <dependency>
        <groupId>org.apache.pdfbox</groupId>
        <artifactId>fontbox</artifactId>
        <version>1.8.0-SNAPSHOT</version>
    </dependency>
    <dependency>
        <groupId>org.apache.pdfbox</groupId>
        <artifactId>pdfbox</artifactId>
        <version>1.8.0-SNAPSHOT</version>
    </dependency>

.

finally the working code:

import com.google.cloud.storage.*;
import org.apache.pdfbox.pdmodel.PDDocument;
import org.apache.pdfbox.util.PDFTextStripper;

public class Download {

    public static String perform(String bucket, String file) throws Exception {
        byte[] fileByte = download(bucket, file);
        String pdfFileTxt = pdf2txt2(fileByte);
        return pdfFileTxt;
    }

    public static byte[] download(String bucketName, String fileId) throws IOException {
        Storage storage = StorageOptions.getDefaultInstance().getService();
        BlobId blobId = BlobId.of(bucketName, fileId);
        Blob blob = storage.get(blobId);
        return blob.getContent();
    }


    public static String pdf2txt2(byte[] byteArr) throws IOException {
        InputStream myInputStream = new ByteArrayInputStream(byteArr);
        PDDocument pddDoc = PDDocument.load(myInputStream);
        PDFTextStripper reader = new PDFTextStripper();
        String pageText = reader.getText(pddDoc);
        pddDoc.close();
        return pageText;
    }
}