A little about OAuth 2.0
Whenever a user signs up to your app/website via Google or 3rd Party, an Authorization Code, this Authorization Code is exchanged for an AccessToken & RefreshToken.
The AccessToken sent via Google are valid generally for 60 minutes.
Offline Access (Server Side)
Let's break it down to two parts:
If your need to update within 60 minutes of user's last activity
You can use firebase along with gapi to achieve that. You'll be provided with the AccessToken that can be sent back to server to add to calendar.
More info on implementation
If you need to update after 60 minutes of user's last activity
Firebase & gapi's most method handle the AuthorizationCode flow internally. They even further refresh the AccessToken after 60 minutes. This is beneficial for most developers as they won't have a headache of managing all the tokens.
This method but, hides RefreshToken & AuthorizationCode from the developer. That is even if your server has the access token, it won't be able to refresh it and it would be deemed useless.
To achieve complete offline access, in the initial request to get AuthorizationCode you will need to send a HTTP GET parameter access_type to offline
GAPI provides you with grantOfflineAccess() method which returns the AuthorizationCode that can be later used on your server to fetch access token & refresh token.
Note: If you are storing AuthorizationCode in your database, make sure it is secure. The limitation in Firebase are set due to security reason. It is more secure to not talk with AuthorizationCode generally.
More links