17

All the git projects for the AOSP are cloned by the repo tool, which reads this xml: https://android.googlesource.com/platform/manifest/+/refs/heads/master/default.xml.

AOSP guide says the in order to build, we should run source build/envsetup.sh on the folder where repo cloned all repositories. So let's look at the platform/build on the default.xml from the manifest repository. We get

  <project path="build/make" name="platform/build" groups="pdk" >
    <copyfile src="core/root.mk" dest="Makefile" />
    <linkfile src="CleanSpec.mk" dest="build/CleanSpec.mk" />
    <linkfile src="buildspec.mk.default" dest="build/buildspec.mk.default" />
    <linkfile src="core" dest="build/core" />
    <linkfile src="envsetup.sh" dest="build/envsetup.sh" />
    <linkfile src="target" dest="build/target" />
    <linkfile src="tools" dest="build/tools" />
  </project>

We confirm where envsetup.sh is located., it is in platform/build. It defines the function m which according to the AOSP guide, builds the entire AOSP project:

function _trigger_build()
(
    local -r bc="$1"; shift
    if T="$(gettop)"; then
      _wrap_build "$T/build/soong/soong_ui.bash" --build-mode --${bc} --dir="$(pwd)" "$@"
    else
      echo "Couldn't locate the top of the tree. Try setting TOP."
    fi
)
function m()
(
    _trigger_build "all-modules" "$@"
)

Ok, so looks like build/soong/soong_ui.bash is the place called when we run the m function, so this script should build everything.

Here's soong_ui.bash. It sources source ${TOP}/build/soong/scripts/microfactory.bash and then calls soong_build_go soong_ui android/soong/cmd/soong_ui

Here's microfactory.bash, where we find function soong_build_go

soong_build_go
{
    BUILDDIR=$(getoutdir) \
      SRCDIR=${TOP} \
      BLUEPRINTDIR=${TOP}/build/blueprint \
      EXTRA_ARGS="-pkg-path android/soong=${TOP}/build/soong -pkg-path github.com/golang/protobuf=${TOP}/external/golang-protobuf" \
      build_go $@
}

We find build_go in microfactory.bash from build/blueprint:

Looks like all of this is for building the microfactory.go project. I think it has something to do with the soong build system.

I'm now lost. After building microfactory.go, what happens? Where does actual Android code gets built?

microfactory.sh says build_go does this: Bootstrap microfactory from source if necessary and use it to build the requested binary. The requested binary is android/soong/cmd/soong_ui

I'm trying to find android/soong/cmd/soong_ui but I don't know what/where it is, but I'd guess is the soong build system, not the AOSP project yet.

UPDATE:

on soong_ui.bash, I noticed it end with

cd ${TOP}
exec "$(getoutdir)/soong_ui" "$@"

Remember that this is called form envsetup.sh. Well, ${TOP}, I guess, is the place where repo clones everything. Looks like it's trying to execute soong_ui with the arguments from envsetup.sh which are --build-mode --${bc} --dir="$(pwd)" "$@", where this $@ is "all-modules" "$@", I guess.

I assume song_ui is the soong executable. It should look for Android.bp on the ${TOP}, but I don't think there is one on the place where repo cloned everything.

oguz ismail
  • 1
  • 16
  • 47
  • 69
Guerlando OCs
  • 1,886
  • 9
  • 61
  • 150

2 Answers2

8

You have already found out a lot, and you are right with the link from m to soong_ui.bash and then starting microfactory.

From my reading of the code, the purpose of soong_build_go is to build the package android/soong/cmd/soong_ui, with the binary name soong_ui. Like Yong said in the other answer, this creates the binary soong_ui under the directory $(getoutdir), and the source for that binary is located at build/soong/cmd/soong_ui/main.go.

As for your updated question about an Android.bp file, it is symlinked from build/soong/root.bp when repo sync is run, but as you can see, the file is empty.

Instead, in m it tells Soong to build all_modules, which eventually runs another tool called kati. From the description in https://github.com/google/kati, Kati processes GNU makefiles and turns them into Ninja build files.

At this point we can (mostly) assume regular Make semantics, even though the underlying build system is actually Kati and Ninja and Soong etc. Since the working directory is $TOP, the Makefile at the root directory, which is symlinked from build/make/core/root.mk is used. Which includes main.mk which then includes build/make/core/Makefile. Inside that makefile you can see how the different .img files are built (e.g. system.img).

Maurice Lam
  • 1,577
  • 9
  • 14
  • Note that it has been a while since I last looked at this code, and the system changed a bit, so it would be best for you to double check what I said above. In particular, I see `build/soong/filesystem/filesystem.go` being checked in very recently, which looks like it may replace some/most of the code path I described above. – Maurice Lam Jan 01 '21 at 04:22
  • I remember reading about Android.bp replacing several makefiles, but it looks like it's almost not used. Soong should build Android.bp files but all it does is to call Kati to build makefiles? – Guerlando OCs Jan 01 '21 at 07:14
  • 1
    Android.bp files are used. From a build system user's perspective, defining a target in .bp is equivalent to defining it in make. If you look inside the source for [build.go](https://cs.android.com/android/platform/superproject/+/master:build/soong/ui/build/build.go;l=176-295;drc=3c9f5ac787263055ed9de38502de5b0a4c142de5), it calls a number of actions, including runSoong, runKati, runNinja, and runBazel. My (unverified) guess is that Soong turns .bp files into ninja, and Kati turns .mk files into ninja, and then Ninja takes both to build the output artifact. I don't know where Bazel fits in. – Maurice Lam Jan 01 '21 at 09:59
  • I can double confirm: Soong parses .bp into .ninja files. It's mentioned in Soong source code. Bazel is introduced only in recent Android versions (11, 12). So it may be not widely used yet. – LVitya Oct 22 '21 at 14:27
4

Let's take make systemimage as an example:

The call sequence is:

  1. prebuilts/build-tools/linux-x86/bin/makeparallel --ninja build/soong/soong_ui.bash --make-mode "systemimage". And $(getoutdir)/soong_ui is build by "build_go soong_ui android/soong/cmd/soong_ui"
  2. build/soong/cmd/soong_ui/main.go#main()
  3. soong/ui/build/build.go#Build()
Yong
  • 1,529
  • 12
  • 21