I'm using PowerShellGet to install my own PowerShell module from my local Nuget feed (packed and published as here).
I'm installing it once like this:
Install-Module MyModule -Scope CurrentUser
And then I'm updating it like this:
Update-Module MyModule
It works as expected, but after every version update I'm getting duplicate of module instead of override:
Get-Module MyModule -ListAvailable
# Output
    Directory: C:\Users\user\Documents\WindowsPowerShell\Modules
ModuleType Version    Name       ExportedCommands
---------- -------    ----       ----------------
Manifest   1.0.0.40   MyModule   {...}
Manifest   1.0.0.39   MyModule   {...}
Manifest   1.0.0.38   MyModule   {...}
Functions which were removed in latest version remain available and PowerShell ISE auto complete shows duplicates for every function:
Update
@CmdrTchort proposed additional checks for investigation, here are results.
Check where modules are installed
Get-Module -Name MyModule -ListAvailable | %{ $_.ModuleBase }
As expected, PowerShellGet installed them to %USERPROFILE%\Documents\WindowsPowerShell\Modules:
C:\Users\user\Documents\WindowsPowerShell\Modules\MyModule\1.0.1.1
C:\Users\user\Documents\WindowsPowerShell\Modules\MyModule\1.0.0.40
C:\Users\user\Documents\WindowsPowerShell\Modules\MyModule\1.0.0.39
C:\Users\user\Documents\WindowsPowerShell\Modules\MyModule\1.0.0.38
I see that $PSScriptRoot contains that path.
Check what Update-Module will do
Update-Module -Name MyModule -WhatIf
# Result
# What if: Performing the operation "Update-Module" on target "Version '1.0.1.1' of module 'MyModule', updating to version '1.0.1.2'".
Check, which version is actually imported:
- Start new 
powershellconsole - Run 
Get-Moduleto ensure that MyModule not imported at all - Run any cmdlet from MyModule
 - Check, that only one, latest version of MyModule was imported by executing 
Get-Moduleagain 
For me it is:
ModuleType Version    Name       ExportedCommands
---------- -------    ----       ----------------
Manifest   1.0.1.2    MyModule   {...
Conclusion
It seems, that there is no issue here, as @CmdrTchort wrote it is expected that Get-Module -ListAvailable shows all versions installed. 
I think that behavior of PowerShell auto-complete in powershell and powershell_ise is a bit confusing though since it shows duplicates of same function if few versions available and functions removed from latest version, probably will be changed soon or somehow configurable.
