How to download and use the universal sentence encoder instead of loading it from url

Question

I am using the universal sentence encoder to find sentence similarity. below is the code that i use to load the model

import tensorflow_hub as hub
model = hub.load("https://tfhub.dev/google/universal-sentence-encoder-multilingual-large/3")

Here, instead of lookin at the url in tensorflow hub, Is there a way to download the model programmatically to a local location and load the model from local filesystem

Hey were you able to download the model? I'm trying to do the same — Chris_007, Sep 05 '22 at 03:56

score 2 · Answer 1 · answered Jul 20 '22 at 05:07

Well i am not sure and i haven't tried it but i checked the source of the hub.load() and i found some interesting facts may be they help you for your problem

First of all the doc says

This function is roughly equivalent to the TF2 function
tf.saved_model.load() on the result of hub.resolve(handle). Calling this function requires TF 1.14 or newer. It can be called both in eager and graph mode.

that means the function can handle both URL or saved model in a file system, to confirm that i checked the documentation of hub.resolve() which is being used internally in hub.load() and there i found some thing of your interest

def resolve(handle):
  """Resolves a module handle into a path.
  This function works both for plain TF2 SavedModels and the legacy TF1 Hub
  format.
  Resolves a module handle into a path by downloading and caching in
  location specified by TF_HUB_CACHE_DIR if needed.
  Currently, three types of module handles are supported:
    1) Smart URL resolvers such as tfhub.dev, e.g.:
       https://tfhub.dev/google/nnlm-en-dim128/1.
    2) A directory on a file system supported by Tensorflow containing module
       files. This may include a local directory (e.g. /usr/local/mymodule) or a
       Google Cloud Storage bucket (gs://mymodule).
    3) A URL pointing to a TGZ archive of a module, e.g.
       https://example.com/mymodule.tar.gz.
  Args:
    handle: (string) the Module handle to resolve.
  Returns:
    A string representing the Module path.
  """
  return registry.resolver(handle)

The documentation clearly says it supports the path to the local file system which points to the module/model files, you should now perform some experiments and give it a try. For more details have a look on this source file

How to download and use the universal sentence encoder instead of loading it from url

1 Answers1