You can convert times and frequencies on fly. You have to use  __getitem__ and probably lru_cache  to store some values for further usage.
Let say that fourier is something like this 
class Fourier():
   def __init__(self,a=10):
      self.a=a
   def __getitem__(self, index): 
      #this is function that calculates and returns value of my_furier
      return self.a+index
t=Fourier()
print(t[12.4])
You can apply same thing for accessing time from Fourier.  So you can create new time object that enables you picking any valid time and returns that time or use some kind of interpolation to return values that are not in table.
If you will not be able to store all values in ram, you can use shelve module from standard library to store and acess items from disk and you can apply interface whit interpolation on it if required.