1

The experiment software sacred was run without MongoDB in the background with a configured mongo-observer. When it tried to write the settings to MongoDB, this failed, creating the file /tmp/sacred_mongo_fail__eErwU.pickle, with the message

Warning: saving to MongoDB failed! Stored experiment entry in /tmp/sacred_mongo_fail__eErwU.pickle
Traceback (most recent calls WITHOUT Sacred internals):
  File "/usr/local/lib/python2.7/dist-packages/sacred/observers/mongo.py", line 127, in started_event
    self.run_entry[experiment][sources] = self.save_sources(ex_info)
  File "/usr/local/lib/python2.7/dist-packages/sacred/observers/mongo.py", line 239, in save_sources
    file = self.fs.find_one({filename: abs_path, md5: md5})
  File "/usr/local/lib/python2.7/dist-packages/gridfs/__init__.py", line 261, in find_one
    for f in self.find(filter, *args, **kwargs):
  File "/usr/local/lib/python2.7/dist-packages/gridfs/grid_file.py", line 658, in next
    next_file = super(GridOutCursor, self).next()
  File "/usr/local/lib/python2.7/dist-packages/pymongo/cursor.py", line 1114, in next
    if len(self.__data) or self._refresh():
  File "/usr/local/lib/python2.7/dist-packages/pymongo/cursor.py", line 1036, in _refresh
    self.__collation))
  File "/usr/local/lib/python2.7/dist-packages/pymongo/cursor.py", line 873, in __send_message
    **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/pymongo/mongo_client.py", line 888, in _send_message_with_response
    server = topology.select_server(selector)
  File "/usr/local/lib/python2.7/dist-packages/pymongo/topology.py", line 214, in select_server
    address))
  File "/usr/local/lib/python2.7/dist-packages/pymongo/topology.py", line 189, in select_servers
    self._error_message(selector))
ServerSelectionTimeoutError: localhost:27017: [Errno 111] Connection refused

How can this pickle file be imported into MongoDB manually?

serv-inc
  • 35,772
  • 9
  • 166
  • 188

1 Answers1

1
  1. Load the pickle file,
  2. set the _id,
  3. insert 

db = pymongo.MongoClient().sacred
entry = pickle.load(open('/tmp/sacred_mongo_fail__eErwU.pickle'))
entry['_id'] = list(db.runs.find({}, {"_id": 1}))[-1]['_id']
db.runs.insert_one(entry)

This is quick and dirty, depends on the find to list objects in order, and could use Cleanest way to get last item from Python iterator instead of list(...)[-1], but it should work.

serv-inc
  • 35,772
  • 9
  • 166
  • 188
  • I feel like this answer is on the right track, but it hasn't worked for me. For starters, my experiment in the pickle file already had an '_id' field, and the open command needs the 'rb' argument to open as binary in python3... but even with that, even though the insert_one command doesn't return an error, it doesn't actually update the (existing) record in the db. Any ideas? – clemej Jan 03 '18 at 19:47
  • @clemej: if the `_id` already exists, you could `find_one_and_update`. See https://stackoverflow.com/a/46608956/1587329 etc. – serv-inc Feb 01 '18 at 13:16