Use a combination of SQLite and XMPP MAM ext.
- Store the new messages in a database.
- Load and display the last x messages from local database.
- XMPP MAM extension will get information from server, from present, past, even if they were not present in the client's memory.
- Check database and the server when the user wants to view more messages.
- Cache eviction policy to reset/remove past messages from database to avoid memory usage.
Retrieve the last 20 messages using XMPP MAM ext:
int n = 20; // The number of messages to retrieve
MamManager mamManager = MamManager.getInstanceFor(connection);
MamQueryResult mamQueryResult = mamManager.queryArchive(n);
List<Forwarded> forwardedList = mamQueryResult.getForwardedMessages();
for (Forwarded forwarded : forwardedList) {
    Stanza stanza = forwarded.getForwardedStanza();
    if (stanza instanceof Message) {
        Message message = (Message) stanza;
        // Display the message to the user or process it as needed
    }
}
connection represents XMPP, n var represents the number of messages and mamManager.queryArchive(n) retrieves the last n message from MAM.
The if (stanza instanceof Messages) checks the Message stanzas as MAM can retrieve it like Presence.
See XEP-0313 for more information.
Open Source examples can be good to check like Smack, SQLite and Pix-Art Messanger.