Make these variables.
private int previousTotal = 0;
private boolean loading = true;
private int visibleThreshold = 5;
int firstVisibleItem, visibleItemCount, totalItemCount;
Set on Scroll for recycler view.
mRecyclerView.addOnScrollListener(new RecyclerView.OnScrollListener() {
    @Override
    public void onScrolled(RecyclerView recyclerView, int dx, int dy) {
        super.onScrolled(recyclerView, dx, dy);
        visibleItemCount = mRecyclerView.getChildCount();
        totalItemCount = mLayoutManager.getItemCount();
        firstVisibleItem = mLayoutManager.findFirstVisibleItemPosition();
        if (loading) {
            if (totalItemCount > previousTotal) {
                loading = false;
                previousTotal = totalItemCount;
            }
        }
        if (!loading && (totalItemCount - visibleItemCount) 
            <= (firstVisibleItem + visibleThreshold)) {
            // End has been reached
            Log.i("Yaeye!", "end called");
            // Do something
            loading = true;
        }
    }
});
Note : Make sure you are using LinearLayoutManager as layout manager for RecyclerView.
LinearLayoutManager mLayoutManager;
mLayoutManager = new LinearLayoutManager(this);
mRecyclerView.setLayoutManager(mLayoutManager);
and for a grid
GridLayoutManager mLayoutManager;
mLayoutManager = new GridLayoutManager(getActivity(), spanCount);
mRecyclerView.setLayoutManager(mLayoutManager);
Have fun with your endless scrolls !! ^.^
Update : mRecyclerView.setOnScrollListener() is deprecated just replace with mRecyclerView.addOnScrollListener() and the warning will be gone! You can read more from this SO question.
Since Android now officially support Kotlin, here is an update for the same -
Make OnScrollListener 
class OnScrollListener(val layoutManager: LinearLayoutManager, val adapter: RecyclerView.Adapter<RecyclerAdapter.ViewHolder>, val dataList: MutableList<Int>) : RecyclerView.OnScrollListener() {
    var previousTotal = 0
    var loading = true
    val visibleThreshold = 10
    var firstVisibleItem = 0
    var visibleItemCount = 0
    var totalItemCount = 0
    override fun onScrolled(recyclerView: RecyclerView, dx: Int, dy: Int) {
        super.onScrolled(recyclerView, dx, dy)
        visibleItemCount = recyclerView.childCount
        totalItemCount = layoutManager.itemCount
        firstVisibleItem = layoutManager.findFirstVisibleItemPosition()
        if (loading) {
            if (totalItemCount > previousTotal) {
                loading = false
                previousTotal = totalItemCount
            }
        }
        if (!loading && (totalItemCount - visibleItemCount) <= (firstVisibleItem + visibleThreshold)) {
            val initialSize = dataList.size
            updateDataList(dataList)
            val updatedSize = dataList.size
            recyclerView.post { adapter.notifyItemRangeInserted(initialSize, updatedSize) }
            loading = true
        }
    }
}
and add it to your RecyclerView like this 
recyclerView.addOnScrollListener(OnScrollListener(layoutManager, adapter, dataList))
For a full code example, feel free to refer this Github repo.