1

I have the following scenario: I have a solution A with multiple projects including some "base" projects and projects (denoted by D) dependent on libraries produced by these base projects (thus projects in D have a project reference to projects in base). I also have two solution filters: filter1.slnf containing base projects, and filter2.slnf containing projects in D.

I am trying to first run msbuild filter1.slnf and then run msbuild filter.slnf /p:BuildProjectReferences=false. If I understand correctly BuildProjectReferences=false is to instruct msbuild to not explicitly build project references/dependencies. The aim being to make the second build step utilize the artifacts from the previous step. Changes to base projects are infrequent and I would not want to build them again and again in the CI/CD pipeline. But I also would like to keep the current solution to make it easier to make those infrequent changes, and thus solution filters.

In the output for the second command I see msbuild is trying to read the base project files.

My question is where should I look in the msbuild's open source-code to verify if it is just doing a simple check of whether the output directory contains the base libraries ? Also is this approach of breaking the build of the complete solution into different builds (eg. one filter for base projects and one/or more filter(s) for the dependent projects) using solution filters viable (as the official blog does not seem to speak explicitly of this use case)?

Related post: How do you split a Visual Studio Solution?

advocateofnone
  • 2,527
  • 3
  • 17
  • 39
  • If only building the "high level" projects, where would the base binaries come from? Hopefully, you haven't committed them to source control? – PMF Apr 15 '23 at 18:30
  • I don't intend to build only "high" level projects. Rather I would have two builds in the same build chain. With the build for "high" level projects having an artifact dependency on the build for "base" projects. So if frequent changes are made only to "high" level projects the artifacts from previous build(s) for base projects would be used. – advocateofnone Apr 15 '23 at 18:35
  • Ok, understood. Sounds reasonable. I don't know the answer to the question, tough. – PMF Apr 15 '23 at 20:19
  • Do you have only one agent for the build or are you forcing the build to always go to the same agent? Do you preserve the agent's workspace across builds (i.e. you don't clean)? If the workspace is cleaned on each build and/or the build is run on different agents then the 'base' projects may be built in the CI pipeline regardless of no new source changes. – Jonathan Dodds Apr 16 '23 at 01:56
  • @JonathanDodds I have multiple agents. No I do not force to run the builds on specific agents. Yes there are some build clean up rules in place, but according to the current ones the workspace is preserved across builds for quite some time. So either the "base" build will be run only when some change are done to base projects, or, if no changes are done then after a"long" enough time. – advocateofnone Apr 16 '23 at 05:58
  • When there is a change to the "base" build, every agent will separately build the change but then the agents will be 'up to date' until the next "base" build change. – Jonathan Dodds Apr 16 '23 at 14:06
  • You are relying on incremental builds which have a risk of 'drift' over time. An incremental build is not easily repeatable because both the state of source control and the state of the workspaces are inputs. There are conditions that can develop that lead to an incremental build being 'untrue' and failing or succeeding incorrectly. If an incremental build is used it would be advisable to schedule a periodic 'clean everything and build everything' build to reset and mitigate the risk of drift. – Jonathan Dodds Apr 16 '23 at 14:12
  • The handling of the `BuildProjectReferences` property is largely in the `Microsoft.Common.CurrentVersion.targets` file. The `build` target is invoked on a `ProjectReference`'d project when the `BuildProjectReferences` property is `true`. There is no check of the output directory related to the state of the `BuildProjectReferences` property. – Jonathan Dodds Apr 16 '23 at 14:21

1 Answers1

1

For greenfield software development, I would advise against using filters to create separate builds in the same pipeline. I would also advise against having separate builds in the same pipeline regardless of how the builds are implemented.

Given the following:

  • Set of base projects (Set B)
  • Set of projects that consume set B (Set D)
  • Source control is git
  • Azure DevOps is used for pipelines

I would suggest:

  • Create two separate repos, one for each set B and one for set D
  • Create separate CI build pipelines for the two sets
  • Create a private NuGet package feed (Azure Artifacts)
  • Create steps for the Set B products to be published as a package in the private NuGet package feed - the steps could be in a separate 'delivery' stage or a separate 'delivery' pipeline
  • projects in Set D should use a PackageReference to the Set B package

Git is very efficient but separate repos still minimize the size of the clone required by a build.

Instead of sharing in source control, sharing is achieved through packages. By using a package for set B, an existing build is essentially kept and re-used for multiple builds of set D.

Currently with NuGet and MSBuild, there is not a way to swap or cascade between PackageReference and ProjectReference. There are situations where that would be nice and there may be future changes to address that.

Set B and D could be in one repo but there should be separate directories with no shared source and the CI triggers in each pipeline should specify the paths to include and/or exclude. There shouldn't be a file that if changed, triggers both builds.

Jonathan Dodds
  • 2,654
  • 1
  • 10
  • 14