How to build static library from the Generated source files using Bazel Build

Question

I am trying to make a following setup run with Bazel. With calling “bazel build” a Python script should generate unknown number of *.cc files with random names, and then compile these into single static library (.a file), all within one Bazel call. I have tried following: one generated file has fixed name, this one is referenced in outs of genrule() and srcs of cc_library rule. The problem is I need all generated files to be built as a library, not only the file with fixed name. Any ideas how to do this? My BUILD file:

py_binary(
    name = "sample_script",
    srcs = ["sample_script.py"],
)
genrule(
    name = "sample_genrule",
    tools = [":sample_script"],
    cmd = "$(location :sample_script)",
    outs = ["cpp_output_fixed.cc"], #or shall also the files with random names be defined here?
)
cc_library(
    name = "autolib",
    srcs = ["cpp_output_fixed.cc"],
    #srcs = glob([ #here should all generated .cc files be named
    #    "./*.cc",
    #    "./**/*.cc",
    #    ])+["cpp_output_fixed.cc"], 
)

Python file sample_script.py:

#!/usr/bin/env python
import hashlib
import time

time_stamp = time.time()

time_1 = str(time_stamp)
time_2 = str(time_stamp + 1)

random_part_1 = hashlib.sha1(time_1).hexdigest()[-4:]
random_part_2 = hashlib.sha1(time_1).hexdigest()[-4:]

fixed_file = "cpp_output_fixed" + ".cc"
file_1 = "cpp_output_" + random_part_1 + ".cc"
file_2 = "cpp_output3_" + random_part_2 + ".cc"

with open(fixed_file, "w") as outfile:
    outfile.write("#include <iostream>"
                   "int main() {"
                   "  std::cout <<'''Hello_world'''    <<std::endl;"
                   "  return 0"
                   "}")

with open(file_1, "w") as outfile:
    outfile.write("#include <iostream>"
                   "int main() {"
                   "  std::cout <<'''Hello_world'''    <<std::endl;"
                   "  return 0"
                   "}")

with open(file_2, "w") as outfile_new:
    outfile_new.write("#include <iostream>"
                   "int main() {"
                   "  std::cout <<'''Hello_world''' <<std::endl;"
                   "  return 0"
                   "}")

print ".cc generation DONE"

Answer 1

[big edit, since I found a way to make it work :)]

If you really need to emit files that are unknown at the analysis phase, your only way is what we internally call tree artifacts. You can think of it as a directory that contains files that will only be inspected at the execution phase. You can declare a tree artifact from Skylark using ctx.actions.declare_directory.

Here is a working example. Note 3 things:

• we need to add ".cc" to the directory name to fool C++ rules that this is valid input
• the generator needs to create the directory that bazel tells it to
• you need to use bazel@HEAD (or bazel 0.11.0 and later)

genccs.bzl:

def _impl(ctx):
  tree = ctx.actions.declare_directory(ctx.attr.name + ".cc")
  ctx.actions.run(
    inputs = [],
    outputs = [ tree ],
    arguments = [ tree.path ],
    progress_message = "Generating cc files into '%s'" % tree.path,
    executable = ctx.executable._tool,
  )

  return [ DefaultInfo(files = depset([ tree ])) ]

genccs = rule(
  implementation = _impl,
  attrs = {
    "_tool": attr.label(
      executable = True,
      cfg = "host",
      allow_files = True,
      default = Label("//:genccs"),
    )
  }
)

BUILD:

load(":genccs.bzl", "genccs")

genccs(
    name = "gen_tree",
)

cc_library(
    name = "main",
    srcs = [ "gen_tree" ]
)

cc_binary(
    name = "genccs",
    srcs = [ "genccs.cpp" ],
)

genccs.cpp

#include <fstream>
#include <sys/stat.h>
using namespace std;

int main (int argc, char *argv[]) {
  mkdir(argv[1], S_IRWXU);
  ofstream myfile;
  myfile.open(string(argv[1]) + string("/foo.cpp"));
  myfile << "int main() { return 42; }";
  return 0;
}

Answer 2

1) List all output files. 2) Use the genrule as a dependency to the library.

genrule(
    name = "sample_genrule",
    tools = [":sample_script"],
    cmd = "$(location :sample_script)",
    outs = ["cpp_output_fixed.cc", "cpp_output_0.cc", ...]
)

cc_library(
    name = "autolib",
    srcs = [":sample_genrule"],
)