I found out, you cannot do it through the elastic beanstalk console (at least not yet). However you can still set it via the eb cli, or aws cli.
Using EB CLI
Basically what we are trying to do is to update the aws:elb:listener setting, you can see the possible settings in the general options docs.
Using the EB CLI is pretty simple. Assuming we already setup the awsebcli tool for our project we can use the eb config command.
It will open up your default terminal editor and allow you to change settings which are written as a YAML file. When you make a change and save it, the eb config cmd will automatically update the settings for your Elastic Beanstalk environment.
You will need to add the following settings to your config file:
aws:elb:listener:443:
  InstancePort: '80'
  InstanceProtocol: HTTP
  ListenerEnabled: 'true'
  ListenerProtocol: HTTPS
  PolicyNames: null
  SSLCertificateId: CERTIFICATE_ARN_HERE
Change the value for CERTIFICATE_ARN_HERE to your AMC Certificates ARN. You can find it in the AWS Certificate Manager console:

IMPORTANT: Your aws:elb:listener:443 setting MUST be placed above the aws:elb:listener:80 setting. Otherwise the environment configuration update will error out.
Using AWS CLI
The same can be accomplished using the general aws cli tools via the update-environment command.
aws elasticbeanstalk update-environment \
--environment-name APPLICATION_ENV --option-settings \
Namespace=aws:elb:listener:443,OptionName=InstancePort,Value=80 \
Namespace=aws:elb:listener:443,OptionName=InstanceProtocol,Value=HTTP \
Namespace=aws:elb:listener:443,OptionName=ListenerProtocol,Value=HTTPS \
Namespace=aws:elb:listener:443,OptionName=SSLCertificateId,Value=CERTIFICATE_ARN_HERE
NOTE: When you update it via either of the methods above, the Elastic Beanstalk console will not show HTTPS as enabled. But the load balancer will, and it will also apply to the Cloudformation template as well get saved into the EB's configuration.