Thanks to Rossen Stoyanchev answer on github  I was manage to solve this by adding interceptor to the inbound channel. Changes needed in the spring-websocket-portfolio demo application is the following:
Change websocket configuration:
public void configureClientInboundChannel(ChannelRegistration registration) {
    registration.setInterceptors(new TopicSubscriptionInterceptor());
}
And the interceptor was something like this:
public class TopicSubscriptionInterceptor extends ChannelInterceptorAdapter {
    private static Logger logger = org.slf4j.LoggerFactory.getLogger(TopicSubscriptionInterceptor.class);
    @Override
    public Message<?> preSend(Message<?> message, MessageChannel channel) {
        StompHeaderAccessor headerAccessor= StompHeaderAccessor.wrap(message);
        if (StompCommand.SUBSCRIBE.equals(headerAccessor.getCommand())) {
            Principal userPrincipal = headerAccessor.getUser();
            if (!validateSubscription(userPrincipal, headerAccessor.getDestination())) {
                throw new IllegalArgumentException("No permission for this topic");
            }
        }
        return message;
    }
    private boolean validateSubscription(Principal principal, String topicDestination) {
        if (principal == null) {
            // Unauthenticated user
            return false;
        }
        logger.debug("Validate subscription for {} to topic {}", principal.getName(), topicDestination);
        // Additional validation logic coming here
        return true;
    }
}