Retrieve Most Recent Entity from Datomic

Question

I'm interested in entities and their timestamps. Essentially, I want a time-sorted list of entities.

To that end, I've composed the following functions:

(defn return-posts
  "grabs all posts from Datomic"
  []
  (d/q '[:find ?title ?body ?slug
         :where
         [?e :post/title ?title]
         [?e :post/slug ?slug]
         [?e :post/body ?body]] (d/db connection)))

(defn get-postid-from-slug
  [slug]
  (d/q '[:find ?e
         :in $ ?slug
         :where [?e :post/slug ?slug]] (d/db connection) slug))

(defn get-post-timestamp
  "given an entid, returns the most recent timestamp"
  [entid]
  (->
   (d/q '[:find ?ts
          :in $ ?e
          :where
          [?e _ _ _]
          [?e :db/txInstant ?ts]] (d/db connection) entid)
   (sort)
   (reverse)
   (first)))

Which I feel must be a hack rooted in ignorance.

Would someone more well-versed in idiomatic Datomic usage chime in and upgrade my paradigms?

benkay · Accepted Answer · 2013-09-29T17:55:12.317

I was bothered by the idea of adding additional timestamps to a database that nominally understands time as a first-class principle and so (after a night of mulling on the approaches outlined by Ulrik Sandberg) evolved the following function:

(defn return-posts
  "grabs all posts from Datomic"
  [uri]
  (d/q '[:find ?title ?body ?slug ?ts
         :where
         [?e :post/title ?title ?tx]
         [?e :post/slug ?slug]
         [?e :post/body ?body]
         [?tx :db/txInstant ?ts]] (d/db (d/connect uri))))

It's idiomatic in Datalog to omit the binding to the transaction ID itself as we typically don't care. In this situation, we very definitely care and in the words of August Lileaas, wish to "traverse the transaction" (there are situations in which we'd want the post creation time, but for this application the transaction time will suffice for ordering entities).

A notable downside to this approach is that recently edited entries will be bumped up in the list. To that end, I'll have to do something later on in order to get their "first appearance" in Datomic for blog-standard post history.

To summarize: I have bound the transaction entity ID per "post" entity ID, and then looked up the transaction timestamp with this function for later sorting.

It's worth noting that, AFAIK, this gives you the instant the current value of :post/title was asserted for the entity and not when the entity was last the subject of a transaction. — spieden, Sep 10 '15 at 23:55
What if the data does not fit in memory? E.g. you want to get, say, 5 most recent entities and not care about the others. — Terminus, Feb 02 '19 at 00:06

score 2 · Answer 2 · answered Sep 28 '13 at 21:41

There isn't really a more elegant way to do this than traversing the transactions. This is why I prefer to have a separate domain specific attribute for timestamps, instead of relying on the transaction timestamps from Datomic. One example where this is necessary is merging: let's say you have a wiki, and you want to merge two wiki pages. In that case, you probably want to control the timestamp yourself, and not use the timestamp from the transaction.

I like to have the attributes :created-at and :changed-at. When I transact new entities:

[[:db/add tempid :post/slug "..."]
 [:db/add tempid :post/title "A title"]
 [:db/add tempid :created-at (java.util.Date.)]
 [:db/add tempid :changed-at (java.util.Date.)]]

Then for updates:

[[:db/add post-eid :post/title "An updated title"]
 [:db/add post-eid :changed-at (java.util.Date.)]]

That way all I have to do is to read out the :created-at attribute of the entity, which will be ready and waiting in the index.

(defmacro find-one-entity
  "Returns entity when query matches, otherwise nil"
  [q db & args]
  `(when-let [eid# (ffirst (d/q ~q ~db ~@args))]
     (d/entity ~db eid#)))

(defn find-post-by-slug
  [db slug]
  (find-one-entity
    '[:find ?e
      :in $ ?slug
      :where
      [?e :post/slug ?slug]]
    db
    slug))

;; Get timestamp
(:created-at (find-post-by-slug db "my-post-slug"))

Retrieve Most Recent Entity from Datomic

2 Answers2