0

This is regarding Splunk Phantom playbook code deployment.

Whenever we create a new playbook, in the repository it creates two files for each playbook (.json, .py). We have 3 different branches associated with one Repository (DEV, QA, PROD).

In our DEV branch, development work would continue and it has around 300 files(JSON and Python files, that means 150 playbooks in Phantom dev GUI).

Now, I don't want to merge all these files to my QA, I just need a few files those undergone changes in DEV needs to push to QA (cause out of 300 files in DEV, we have around 160 files in QA with same code).

When any changes would happen to DEV files, I have to pick those specific files and raise the pull request then merge (that means only those files with changes in DEV should move to QA {kinda Cherry-picking}).

I have followed below process, but I'm getting merge conflicts.

Process:

  1. Create a new branch

    git checkout -b branch_name

  2. Remove all files for this branch

    git rm -rf .

  3. Commit changes

    git commit -m "create new branch"

  4. Retrieve some files from master branch

    git checkout master -- file1 file2 file3 file4

  5. Commit changes

    git commit -m "create new branch"

After doing this process, I navigate to my VSTS UI, and apply cherry-pick on "commit" which I have done at 4th, 5th steps (I am able to successfully create commit with my required files and at the time of performing cherry-pick, I'm running into issues).

Below is a screenshot when I tried to do the cherry-pick - error message looks like this:

There were conflicts when cherry-picking commit 60625f. This operation needs to be done locally

Josh Gust
  • 4,102
  • 25
  • 41
PS036
  • 1
  • 2
  • Why are you removing all of the files? Just make a new branch off of QA, then cherry pick the changes from DEV that need to go into QA. When you're done, merge that branch to QA. Or revisit your branching model entirely, because this style of code-promotion branching model is more suited to centralized version control and doesn't translate well to Git, as you're seeing. – Daniel Mann Feb 06 '20 at 19:00
  • @Daniel Mann, thanks for your reply 1) why i am deleting all of files ? --- cause here i am making two commits , in first commit i am removing all files and in second commit , just keeping my required files and checkout to my intermediate branch, so that i can do cherry-pick on my second commit with my required files am i going in correct direction ? – PS036 Feb 06 '20 at 19:18
  • Why do you have files in dev that are not intended to be merged into the trunk? The fact that this is so difficult and requires you to do that kind of "branch sanitizing" is a **STRONG** indicator that [you're not using the tool the way it was intended](https://www.ncregister.com/images/uploads/Pope-COMPENDIUM.jpg). You shouldn't have files in a git branch that never make it to the downstream branches. Use `.gitignore` or tell devs to be smarter about what files are actually staged in their commits, so that you can just cherry-pick the commit. – Josh Gust Feb 06 '20 at 21:10
  • Step 4. says you're checking out `master`. What is the relationship between `master` in your local repo and `Dev` on the remote? If `master` is tracking `origin/DEV`, what is tracking `origin/QA` (and `origin/PROD`)? – Josh Gust Feb 06 '20 at 21:20

1 Answers1

1

tl;dr

If you find yourself using cherry-pick or rebase -i every time you want to promote changes through a Pull Request, your process and tools configuration are wrong.

Consider using feature branches

Feature (or topic if you read the git scm book) branches that extend from the trunk (master) are focused and disposable. Not only does this model make Pull Requests and releases easier to understand it's also much easier to manage than keeping 2 or 3 lifetime branches in and out of sync at the appropriate time in the appropriate rhythm.

Be truthful about what needs to be under version control

Having files in one lifetime branch of a repo that are never meant to be brought into the other lifetime branches is a strong indication that those files don't really need to be under version control. While these files may be necessary to develop your product they clearly don't represent real portions of your deliverable. This is exactly the reason the .gitignore file exists. Setup your .gitignore file to allow those files to live in the work-space without being tracked.

Implementing the .gitignore file properly should eliminate the need to do the work that is getting you into merge conflicts in the first place, and using feature branches off one lifetime branch as opposed to 3 lifetime branches will make everything else about implementing DevOps practices for your team and code much easier to understand and maintain.

With that said...

Fixing the general issue of resolving merge conflicts locally

These suggestions are assuming that your PR is trying to put changes from branch_name into origin\QA

Azure devops (formerly VSTS) is asking you to resolve the merge conflicts locally and push those changes to your PR.

After your step 5. you need a local copy of the PR target branch (I'm guessing QA) from the remote (usually identified as origin in your local repo).

Once you've done that, get back to branch_name and run git merge localQA. This should show you that the merge cannot be completed b/c of the conflicts. Now you need to open the files listed in git status that are not added to the index and correct the conflict. When the file conflicts are corrected, you can commit the merge and push.

To take a different approach to your current situation

Don't delete all the files in branch_name and checkout selectively from master, which I assume represents origin/DEV. Instead, cut your branch_name from an up-to-date local copy of origin\QA. This will get you started with a QA copy of the files you want. Then from that branch you can git checkout master -- fileA etc. to rewrite those changes into your QA copy. Now you can push the QA copy to origin and create a PR.

If you're savvy you may have already noticed that this way of doing things is exactly like having a single lifetime branch (QA) that uses PRs to get changes from feature branches. This is because the commits from DEV don't ever actually make it out of DEV.

When you use git checkout <tree-ish> -- [pathspec] you are not moving a commit you are moving changes. For example..

Say I have a branch master and create a branch test for some changes and add a file. enter image description here enter image description here

Now, if I create a new branch and use the checkout <tree-ish> -- [pathspec] you'll see that it is creating a new set of changes for me to commit.

enter image description here

In my example of how you're using version control, if master represents your origin/QA and test represents origin/DEV, why should I even bother with the test branch at all if I'm going to create new commit(s) on newTest that represent the changes made in test? I (you) sholudn't.

An additional alternative

Use git rebase -i on your branch to SPLIT THE COMMITS so that the files you want to ignore are in a commit you can ignore. With this approach, you should still create the branch from origin/QA or whatever your PR target is. Then you can use cherry-pick on the commit(s) you want from the development branch and use a PR between this branch and the target.

Understand: using Rebase in this way is rewriting the history of your branch. Some would argue this is unwise, so be sure you know what you're doing

Josh Gust
  • 4,102
  • 25
  • 41