In Julia, how to create a macro consisting of several optional macros?

Question

In Julia, I am trying out different parallelization libraries, to make my program more performant, and to check if memory consumption is the same as with no parallelization. The unfortunate effect of this is a lot of duplication.

Is there a way to organize my code so that I write the algorithm only once and then some macro with a parameter decides how the code is parallelized? My question is similar to this one. For example, my MWE

using ThreadsX, Folds, FLoops, Polyester
create_data = (n,s) -> [rand(1:n,r) for j=1:n for r∈[rand(1:s)]]

function F!(method ::Int, L ::Vector{Vector{Int}}) ::Nothing
    n = length(L)
    if method==0                  for j=1:n sort!(L[j]) end end
    if method==1 Threads.@threads for j=1:n sort!(L[j]) end end
    if method==2 ThreadsX.foreach(1:n) do j sort!(L[j]) end end
    if method==3 Folds.foreach(1:n)    do j sort!(L[j]) end end
    if method==4 FLoops.@floop    for j=1:n sort!(L[j]) end end
    if method==5 Polyester.@batch for j=1:n sort!(L[j]) end end 
    return nothing end
for mtd=0:5
    L = create_data(10^6,10^3);   
    @time F!(mtd,L) end

returns

 17.967120 seconds
  4.537954 seconds (38 allocations: 3.219 KiB)
  4.418978 seconds (353 allocations: 27.875 KiB)
  5.583201 seconds (54 allocations: 3.875 KiB)
  5.542852 seconds (53 allocations: 3.844 KiB)
  4.263488 seconds (3 allocations: 80 bytes)

so there are different performances already for a very simple problem. In my actual case, instead of sort!(L[j]) I have lots of intensive code with several Arrays, Vector{Vector}s, Dicts, ..., where different threads read from occasionally the same place, but write to different places, allocate space in memory, mutate the input, etc. Is there a way to create a new macro @Parallel so that my code would be just

function F!(method ::Int, L ::Vector{Vector{Int}}) ::Nothing
    n = length(L)
    @Parallel(method) for j=1:n sort!(L[j]) end
    return nothing end

Note that I have never created a macro, I only used them thus far, so some explanation would be welcome.

Answer 1

A macro-based solution is possible, but seems unnecessary to me. I'd rather organize code like this:

function testfun!(x::Vector{Int})
    # here goes the repetitive part
    return sort!(x)
end

# entry point for dispatch, and set up data
function F(kernel!, M, N, strategy::Symbol)
    L = create_data(M, N)
    return F!(kernel!, L, Val{strategy}())
end

# and a function to run the loop for every strategy
function F!(kernel!, L, ::Val{:baseline})
    n = length(L)
    for j=1:n kernel!(L[j]) end
end
function F!(kernel!, L, ::Val{:Threads})
    n = length(L)
    Threads.@threads for j=1:n kernel!(L[j]) end
end
function F!(kernel!, L, ::Val{:ThreadsX})
    n = length(L)
    ThreadsX.foreach(1:n) do j kernel!(L[j]) end 
end
# ...
# + rest of the loop functions for Floops, Folds, etc.

function dotests()
    for strategy = (:baseline, :Threads, :ThreadsX, ...)
        @benchmark F(testfun!, 10^6, 10^3, strategy) 
    end
end

This is showing a dispatch-based approach. You could equally well use dictionaries or conditions. The important point is separating the "runner" F! from the "kernel" function.