Reading another Clojure program as a list of S-Expressions

Question

Suppose I have a very simple .clj file on disk with the following content:

(def a 2)
(def b 3)
(defn add-two [x y] (+ x y))
(println (add-two a b))

From the context of separate program, I would like to read the above program as a list of S-Expressions, '((def a 2) (def b 3) ... (add-two a b))).

I imagine that one way of doing this involves 1. Using slurp on (io/file file-name.clj) to produce a string containing the file's contents, 2. passing that string to a parser for Clojure code, and 3. injecting the sequence produced by the parser to a list (i.e., (into '() parsed-code)).

However, this approach seems sort of clumsy and error prone. Does anyone know of a more elegant and/or idiomatic way to read a Clojure file as a list of S-Expressions?

Update: Following up on feedback from the comments section, I've decided to try the approach I mentioned on an actual source file using aphyr's clj-antlr as follows:

=> (def file-as-string (slurp (clojure.java.io/file "src/tcl/core.clj")))
=> tcl.core=> (pprint (antlr/parser "src/grammars/Clojure.g4" file-as-string))
{:parser
 {:local
  #object[java.lang.ThreadLocal 0x5bfcab6 "java.lang.ThreadLocal@5bfcab6"],
  :grammar
  #object[org.antlr.v4.tool.Grammar 0x5b8cfcb9 "org.antlr.v4.tool.Grammar@5b8cfcb9"]},
 :opts
 "(ns tcl.core\n  (:gen-class)\n  (:require [clj-antlr.core :as antlr]))\n\n(def foo 42)\n\n(defn parse-program\n  \"uses antlr grammar to \"\n  [program]\n  ((antlr/parser \"src/grammars/Clojure.g4\") program))\n\n\n(defn -main\n  \"I don't do a whole lot ... yet.\"\n  [& args]\n  (println \"tlc is tcl\"))\n"}
nil

Does anyone know how to transform this output to a list of S-Expressions as originally intended? That is, how might one go about squeezing valid Clojure code/data from the result of parsing with clj-antlr?

Please, check if http://stackoverflow.com/a/24576026/597473 doesn't already answers your question. — Piotrek Bzdyl, Oct 11 '16 at 12:20
@PiotrekBzdyl Thank you for pointing out the prior question. I would never have found that myself. When I have time, I'll submit an answer based on the lead, but feel free to do so yourself. Since the prior question seems to be left a bit open, I'll leave my question up for the time being. — David Shaked, Oct 11 '16 at 12:33
This approach seems to be pretty straightforward for me. Why do you think it is "clumsy and error prone"? — OlegTheCat, Oct 11 '16 at 13:33
@OlegTheCat I suppose trusting a parser from outside of the core on arbitrary code seems optimistic. If you feel otherwise, I'll be happy to share my code once I consider the options on the table. Also, please share a trust-worthy parser if you know of one. — David Shaked, Oct 11 '16 at 14:09
@PiotrekBzdyl The accepted answer from the previous post doesn't seem to work as expected: For instance, calling the defined function, read-string-safely, on the result of (slurp "sample.clj") seems only to return the first line of the code, (def a 2). — David Shaked, Oct 11 '16 at 14:16

Michiel Borkent · Accepted Answer · 2016-10-12T15:28:36.470

(import '[java.io PushbackReader])
(require '[clojure.java.io :as io])
(require '[clojure.edn :as edn])

;; adapted from: http://stackoverflow.com/a/24922859/6264
(defn read-forms [file]
  (let [rdr (-> file io/file io/reader PushbackReader.)
        sentinel (Object.)]
    (loop [forms []]
      (let [form (edn/read {:eof sentinel} rdr)]
        (if (= sentinel form)
          forms
          (recur (conj forms form)))))))

(comment
  (spit "/tmp/example.clj"
        "(def a 2)
(def b 3)
(defn add-two [x y] (+ x y))
(println (add-two a b))")

  (read-forms "/tmp/example.clj")
  ;;=> [(def a 2) (def b 3) (defn add-two [x y] (+ x y)) (println (add-two a b))]
)

score 2 · Answer 2 · answered Oct 12 '16 at 13:24

Do you need something like this?

(let [exprs (slurp "to_read.clj")]
  ;; adding braces to form a proper list
  (-> (str "("  (str  exprs")"))
  ;; read-string is potentially harmful, since it evals the string
  ;; there exist non-evaluating readers for clojure but I don't know
  ;; which one are good 
      (read-string)
      (prn)))

Reading another Clojure program as a list of S-Expressions

2 Answers2