1

I develop a programming language used for a domain-specific problem, and I wanted to ask a question about compiler building: if you create a compiler that intends to generate straight machine code, then do you need to study Assembly in order to implement such a compiler?

If no, what are the alternatives how said compiler can produce binary executables?

Владислав Король
  • 157
  • 2
  • 10
  • 2
    You can write a compiler in any programming language. Generating an executable is a matter of generating the correct machine code (which differs per operating system) – UnholySheep Aug 01 '22 at 20:17
  • 2
    You *can* write a compiler that generates assembly code and feeds that into an assembler, but it is not the only way – UnholySheep Aug 01 '22 at 20:18
  • 1
    The other popular option is to generate code in some other programming language (e.g. C or LLVM bitcode) and have another compiler translate that into machine code. – fuz Aug 01 '22 at 20:30

2 Answers2

3

If performance is important, you probably don't want to try to generate assembly yourself, unless your domain-specific problem is very simple and specific. Generating efficient asm is much harder than just generating working asm. In a compiler like GCC, optimization passes are more than half the code-base, more than parsing C or even C++.

Generate something that an existing optimizer like LLVM can deal with, like LLVM-IR. Write a portable front-end for your language, leave the target-specific stuff and optimization to LLVM, or to GCC's back-end. https://llvm.org/docs/tutorial/MyFirstLanguageFrontend/index.html has a tutorial.

Of course, to debug your compiler, you may want to learn some assembly to at least know where to start looking in the IR for wrong-code bugs. And of course you'd have to learn LLVM-IR, which is essentially an assembly language.

Or compiling to C is an old-school technique but still works: optimizing C compilers are widely available. (Historically well know CFortran, and C++ was originally implemented with CFront which compiled it to C.)

Depending on your domain-specific problem, you might choose to compile to some other high-level language that matches your problem domain. Pick a language that you can target easily, and that has a good optimizing compiler or JIT run-time. e.g. Julia is reputedly good for number-crunching, I think letting you take advantage of parallelism.

C++ could be a good target if some of its template library functions work well. Ahead-of-time C++ compilers will make an executable that just depends on some libraries, not a "runtime" like a JVM or something. And can compile to a library you can easily call from most other things: C and C++ foreign-function interfaces are common in most other language. Depending on your use-case, this may be important.

This method will let you use a C, C++, Julia, or whatever debugger to see what the code your compiler generated is doing. So you only need to know that target language.

Understanding assembly concepts can be useful to understand what C undefined behaviour might produce the symptoms you're seeing, in case of compiler bugs like out-of-bounds array access. But with modern tools like clang -fsanitize=undefined, you can check for many such problems to help verify your compiler.

Also related: Learning to write a compiler

Peter Cordes
  • 328,167
  • 45
  • 605
  • 847
  • 1
    One example of a (much used) compiler targetting C++ is Unity's IL2CPP compiler, which is used to translate the C# game code into C++ in order to get it to run on platforms that don't allow JIT compilation (such as iOS) – UnholySheep Aug 01 '22 at 20:44
  • Thanks a lot for your detailed answer! If the compiler's job is to produce the executable file in machine language, then how will C++ do this alone? Will it needs to produce a C++ analogue and then compile it or is there a straight way how to get the machine code? This is the main point I struggle to understand. – Владислав Король Aug 02 '22 at 15:47
  • @ВладиславКороль: If you compile your language to C++, then you feed that C++ to a C++ compiler. It makes an executable with machine code. Your compiler can just create *source code* for a working program in some convenient language. (Then you might call it a code-generator rather than compiler proper, but you'll still be solving the actual problem you have.) – Peter Cordes Aug 02 '22 at 18:10
  • @PeterCordes So does that mean if I don't use Assembly in the compiler construction, I will inevitably need to compile it to another language and then compile it? – Владислав Король Aug 02 '22 at 18:16
  • @ВладиславКороль: If you compiled to assembly, you'd still need an assembler + linker to make an executable or library. Not technically another compiler, just another tool. But yes, if you compile to another language, of course you need a compiler for that language to make a final executable. Or some other way to run programs in that language, like an interpreter, but for speed reasons normally you'd choose a target language that has an ahead-of-time compiler. Or a good JIT-compiling run-time. – Peter Cordes Aug 02 '22 at 18:21
  • @ВладиславКороль: If you compile to LLVM-IR, you don't need a *separate* compiler, you just call LLVM library functions to have it compile your data-structures into an object file. (Which you can then link into a library or executable.) Basically the same if you write a GCC front-end that uses GCC's back-end, although in that case GCC will make an actual assembly text file and run GAS on it. – Peter Cordes Aug 02 '22 at 18:22
  • @PeterCordes so is LLVM a magical wand that produces executables from description that you provide it? – Владислав Король Aug 02 '22 at 18:33
  • @ВладиславКороль: It's an optimizing compiler implemented as a library. That compiler includes machine-code generation, unlike GCC where they leave that to a separate assembler. But no, it doesn't produce linked executables, only object files (`.o`) like `gcc -c` or `clang -c` would make. You need `ld` or some other linker to make an actual executable. But that's relatively easy. And I think the LLVM project includes a separate linker. – Peter Cordes Aug 02 '22 at 18:37
  • Thanks a lot for your assistance, you really helped me to understand how compilers work! – Владислав Король Aug 02 '22 at 18:42
2

In our conversation with Peter Cordes, we found the answer. When our purpose is to produce runnable machine code in the process of compilation, there are 4 primary ways to achieve it:

  1. Compile down to Assembly and then assemble it.
  2. Compile it to another language like C/C++ and then compile it separately.
  3. Use some metaprogramming tricks to compile an app once that runs different code as you feed it, basically interpreter with the optimised code installed.
  4. Use external frameworks. One of the solutions is to use Low Level Virtual Machine project that produces object code from the Intermediate Representation (LLVM-IR), and then your task would be to design front-end for your language to convert to the LLVM's language, and use a linker to get an executable from the object code that it provides.

I hope this answer will be useful for those who will stumble across this misunderstanding and find our solutions useful.

Sep Roland
  • 33,889
  • 7
  • 43
  • 76
Владислав Король
  • 157
  • 2
  • 10