If this answer helps you, please leave a comment on what specific advice was helpful!
I recently upgraded from Angular 8 to Angular 13. My build time was comparable (~10 minutes in both Angular 8 and Angular 13), but I struggled with my rebuild time (i.e. I use --watch while I develop; if I change only a few files, Angular 8 was recompiling in <10 seconds; Angular 13 required another 10 minutes!).
I used angular.json (see below for a screenshot) to specify "aot":false and I saw much better rebuild times (back to ~8-10 seconds) -- this option is the same as angular-cli ng build command's --aot=false.
I'm adding my answer because currently no other answer suggests trying to use "aot:false. In fact, many of the most popular answers explicitly suggest trying "aot":true! Those answers aren't wrong (in fact, they identify options that improve performance, like "sourceMap":false.) But for me, "aot":false was the gamechanger.
Also, I recognize the benefits of "AOT" (Ahead-of-Time compilation), for example "faster rendering in the browser".  I still use "AOT" ("aot":true) for my production builds; but my answer here is focused on development, where I use --watch to re-build on file changes, so I want a fast re-build, and I don't mind "slower rendering in the browser".
Some helpful resources (I will add more if I find more):
- This comment suggests AOT compilation might be slower than JIT compilation, which may be the reason 
"aot":false works for me 
- This comment might help you profile your build process (to find the bottleneck/troubleshoot long build times) using 
node --inspect and chrome://inspect 
- This comment has more tips on profiling with 
NG_BUILD_PROFILING=1 
Other tips that may improve build performance:
Some related stuff (more experimental / may not be helpful) :
- Blogposts like this one suggest you could use your own builder, like 
esbuild, to see build performance improvements -- but this is more experimental and may not support "native ways" of doing things in Angular (i.e. the templateUrl property). Also they could come at a cost (i.e. larger bundle sizes) 
- This video from Google Chrome Developers talks about principles for "startup performance" , however it's >1 year old, and focuses bundle size and other best practices; so the video may make suggestions that decrease build performance (for other tradeoffs like smaller bundles or easier debugging).
 
EDIT
I set "aot":false` in my angular.json as shown in the below screenshot. Notice the location/path in the JSON is:
root>projects>{craft-app}>architect>build>configurations>{development}
... where {craft-app} is the specific name of the angular project you need to build and {development} is the specific name for the configuration you want to use and specify in the -c argument, as in:
ng build craft-app -c=development
