How to avoid an exception Prematurely reached end of stream using mongoDB Java driver 3.4+ or 3.6+? (during insertion)

Question

I'm trying to insert some documents to capped collection using this snippet of code:

            // get document with specific fields
            Document found = collection.find().first();
            String getTitle = (String) found.get("title");
            String getUrl = (String) found.get("url");
            String getImg = (String) found.get("img");
            String getPrice = (String) found.get("price");

            // document which I want to get as new
            Document doc = collection.find(new Document("title", getTitle)
                    .append("url", getUrl)
                    .append("img", getImg)
                    .append("price", getPrice)
                    .append("sent", true)).first();

            // if the document doesn't exist, then insert as new
            if (doc == null) {
             collection.insertOne(new Document("title", getTitle)
                   .append("url", getUrl)
                   .append("img", getImg)
                   .append("price", getPrice)
                   .append("sent", true));
        }

It means - rewriting documents. I'm inserting new documents with one more field instead of older docs, because capped collection doesn't allow to update existing document with another size. Since I get an exception: Cannot change the size of a document in a capped collection.

Older doc looks like:

New will be as:

And this code works fine, but after a while (during insertion) I'm getting constantly an error:

com.mongodb.MongoSocketReadException: Prematurely reached end of stream
    at com.mongodb.connection.SocketStream.read(SocketStream.java:88)
    at com.mongodb.connection.InternalStreamConnection.receiveResponseBuffers(InternalStreamConnection.java:491)
    at com.mongodb.connection.InternalStreamConnection.receiveMessage(InternalStreamConnection.java:221)
    at com.mongodb.connection.CommandHelper.receiveReply(CommandHelper.java:134)
    at com.mongodb.connection.CommandHelper.receiveCommandResult(CommandHelper.java:121)
    at com.mongodb.connection.CommandHelper.executeCommand(CommandHelper.java:32)
    at com.mongodb.connection.InternalStreamConnectionInitializer.initializeConnectionDescription(InternalStreamConnectionInitializer.java:83)
    at com.mongodb.connection.InternalStreamConnectionInitializer.initialize(InternalStreamConnectionInitializer.java:43)
    at com.mongodb.connection.InternalStreamConnection.open(InternalStreamConnection.java:115)
    at com.mongodb.connection.UsageTrackingInternalConnection.open(UsageTrackingInternalConnection.java:46)
    at com.mongodb.connection.DefaultConnectionPool$PooledConnection.open(DefaultConnectionPool.java:381)
    at com.mongodb.connection.DefaultConnectionPool.get(DefaultConnectionPool.java:96)
    at com.mongodb.connection.DefaultConnectionPool.get(DefaultConnectionPool.java:82)
    at com.mongodb.connection.DefaultServer.getConnection(DefaultServer.java:72)
    at com.mongodb.binding.ClusterBinding$ClusterBindingConnectionSource.getConnection(ClusterBinding.java:86)
    at com.mongodb.operation.OperationHelper.withConnectionSource(OperationHelper.java:237)
    at com.mongodb.operation.OperationHelper.withConnection(OperationHelper.java:212)
    at com.mongodb.operation.FindOperation.execute(FindOperation.java:482)
    at com.mongodb.operation.FindOperation.execute(FindOperation.java:79)
    at com.mongodb.Mongo.execute(Mongo.java:772)
    at com.mongodb.Mongo$2.execute(Mongo.java:759)
    at com.mongodb.FindIterableImpl$FindOperationIterable.first(FindIterableImpl.java:207)
    at com.mongodb.FindIterableImpl.first(FindIterableImpl.java:148)
    at project.Bot.onUpdateReceived(Bot.java:347)

As I correctly understand, the error appears on the line (maybe simply formatting problem):

 Document found = collection.find().first();

I researched and tried to solve the error using this code like (I'm using free Tier M0 cluster):

  MongoCredential credential = MongoCredential.createCredential("admin1", "admin", "mypassword".toCharArray());
        MongoClientSettings settings = MongoClientSettings.builder()
                .credential(credential)
                .retryWrites(true)
                .applyToConnectionPoolSettings(builder ->
                        builder.maxConnectionIdleTime(60000, TimeUnit.MILLISECONDS))
                .applyToSocketSettings(builder ->
                        builder.keepAlive(true))
                .applyToSslSettings(builder -> builder.enabled(true))
                .applyToClusterSettings(builder -> {
                    builder.hosts(Arrays.asList(
                            new ServerAddress("cluster0-shard-00-00-ox90k.mongodb.net", 27017),
                            new ServerAddress("cluster0-shard-00-01-ox90k.mongodb.net", 27017),
                            new ServerAddress("cluster0-shard-00-02-ox90k.mongodb.net", 27017)
                    ));
                    builder.requiredReplicaSetName("Cluster0-shard-0");
                })
                .build();

        MongoClient mongoClient = MongoClients.create(settings);

The same error: com.mongodb.MongoSocketReadException: Prematurely reached end of stream

Updated: Tried instead of rewriting full document, to change just one field like:

Document found = database.getCollection("capped_collection").find(new Document("title", title)
                .append("url", url)
                .append("img", img)
                .append("price", price)).first();

        if (found == null) {
            collection.insertOne(new Document("title", title)
                            .append("url", url)
                            .append("img", img)
                            .append("price", price)
                            .append("sent", false));

    Document doc = collection.find(eq("sent", false)).first();

    if(doc != null){
     collection.updateOne(eq("sent", false), new Document("$set", new Document("sent", true)));
    }

But still have:

com.mongodb.MongoSocketReadException: Prematurely reached end of stream

Tried to change version , the result is here.

Does anyone know what I need to correct or how to solve the error Prematurely reached end of stream?

I appreciate any help.

Some clarification required: 1) If you need to modify the documents after insertion, do you really need capped collection, or do you need a TTL index? (see https://docs.mongodb.com/manual/core/index-ttl/). 2) "Prematurely reached end of stream" is typically a server/network issue, which is a different set of problems vs. document modification in a capped collection. Which question are you trying to ask? 3) Atlas should have an option to show you the correct connection string. Have you tried using it in `MongoClient.create()` instead of using `new MongoClient()`? — kevinadi, Oct 03 '18 at 23:07
Regarding capped collection: you cannot modify a document in a capped collection if it will change the size of the document. However, if it doesn't change the size, you can modify it. For example, if you insert the original document with `{_id:.., title:..., uri:..., img:..., price:..., sent:false}`, if you subsequently do an update e.g. `db.collection.updateOne({_id:..}, {$set: {sent:true}})` it will not change the document size (only setting `false` -> `true`) and the update will succeed. — kevinadi, Oct 03 '18 at 23:12
@KevinAdistambha, "Prematurely reached end of stream" is thrown in process of insertion, so I can't insert all new Documents instead of older. I suppose that I need to modify my connection string using some [options](https://docs.mongodb.com/manual/reference/connection-string/#connection-string-options). — invzbl3, Oct 04 '18 at 00:14
1) Capped collection is useful, but thanks for advice with TTL, I'll check; 2) I'm trying to undertand how to avoid the error during insertion. Because now I can't insert all new documents because of this error just part of them. 3) No, I will check it with `MongoClient.create():`, thanks — invzbl3, Oct 04 '18 at 00:42
Re: connection string, if you press the "Connect" button in Atlas, then "Connect your application", "Driver 3.6 or later", it will show you the connection string you need. In my case it shows `mongodb+srv://kevinadi:@cluster0-wsdgm.mongodb.net/test?retryWrites=true`. I dropped that string into a test Java application using `MongoClients.create(...)` and it can connect to Atlas. — kevinadi, Oct 04 '18 at 00:53
@KevinAdistambha, undertand. If I want to use new format, I need to write MongoClients.create(...) instead of new MongoClient(). Because MongoClient() doesn't support 3.6 driver connection. — invzbl3, Oct 04 '18 at 01:19
@KevinAdistambha, tell me, please, if I use java 8, can I use java driver 3.7+? It's important for me in project. — invzbl3, Oct 04 '18 at 01:22
Yes you can. See [Java driver language compatibility list](https://docs.mongodb.com/ecosystem/drivers/driver-compatibility-reference/#reference-compatibility-language-java) for the full matrix. — kevinadi, Oct 04 '18 at 01:25
@KevinAdistambha, thanks to your pointing on `MongoClient.create()` I finally connected via driver version 3.7.1. with the same code and still the error appears `Prematurely reached end of stream.` — invzbl3, Oct 05 '18 at 19:53
@KevinAdistambha, also I tried as you said me, with `updateOne` like: `Document doc = collection.find(eq("sent", false)).first();` `if(doc != null){ collection.updateOne(eq("sent", false), new Document("$set", new Document("sent", true))); }` Yes, it works, but I get also `com.mongodb.MongoSocketReadException: Prematurely reached end of stream` after a while during insertion. — invzbl3, Oct 06 '18 at 10:21

invzbl3 · Accepted Answer · 2021-06-18T16:08:00.207

Option #1

The error was solved by the format of connection (using the parameters maxIdleTimeMS, ssl and authSource):

maxIdleTimeMS - the maximum number of milliseconds that a connection can remain idle in the pool before being removed and closed.

MongoClient mongoClient = MongoClients.create("mongodb://user:mypassword@cluster0-shard-00-00-ox90k.mongodb.net:27017,cluster0-shard-00-01-ox90k.mongodb.net:27017,cluster0-shard-00-02-ox90k.mongodb.net:27017/test?ssl=true&replicaSet=Cluster0-shard-0&authSource=admin&retryWrites=true&maxIdleTimeMS=5000");

Or you can configure the credentials programmatically (using java driver 3.6+ versions):

admin - is the database where the users are defined in Atlas;

user - is the user name;

mypassword - is the password;

MongoCredential credential = MongoCredential.createCredential("user", "admin", "mypassword".toCharArray());
        MongoClientSettings settings = MongoClientSettings.builder()
                .credential(credential)
                .retryWrites(true)
                .applyToConnectionPoolSettings(builder ->
                        builder.maxConnectionIdleTime(5000, TimeUnit.MILLISECONDS))
                .applyToSslSettings(builder -> builder.enabled(true))
                .applyToClusterSettings(builder -> {
                    builder.hosts(Arrays.asList(
                            new ServerAddress("cluster0-shard-00-00-ox90k.mongodb.net", 27017),
                            new ServerAddress("cluster0-shard-00-01-ox90k.mongodb.net", 27017),
                            new ServerAddress("cluster0-shard-00-02-ox90k.mongodb.net", 27017)
                    ));
                    builder.requiredReplicaSetName("Cluster0-shard-0");
                })
                .build();

        MongoClient mongoClient = MongoClients.create(settings);

When defining the host names, mention all the replica set host names.

This will resolve the java.net.UnknownHostException.

You can't use the DNS short name as with the mongodb+srv connection.

Option #2

Moreover, the error can be solved by calling mongodb.MongoClient.connect once, not each request. Try to restructure your code to call connection once instead of each time during insertion of specific document. In this case, you can avoid any additional parameters from option #1.

Just enough:

mongodb+srv://admin:password@cluster0-ox90k.mongodb.net/test?retryWrites=true&w=majority

Special thanks for help mongoDB support.

kish · Answer 2 · 2021-10-08T23:08:10.470

2

Please check your IP is whitelisted when you try to connect to your free cluster on Atlas. Go to 1) Add connection IP address and click on IP Access List tab there you can add IP address.

edited Oct 08 '21 at 23:08

answered Oct 08 '21 at 22:58

kish

41
3

How to avoid an exception Prematurely reached end of stream using mongoDB Java driver 3.4+ or 3.6+? (during insertion)

2 Answers2

Option #1

Option #2

Linked