Bitbucket Append build number to package.json during pipeline execution

Question

I've an Angular project on a Bitbucket repository. I created this pipeline that deploys the application on AWS S3 and invalidate the CloudFront distribution. Everything works fine. I would like to add the build number to the published version in order to know not just the version of the application but also the build that generated it.

image: cypress/browsers:node14.17.0-chrome88-ff89
options:
  max-time: 120

pipelines:
  branches:   
    master:
      - step:
          runs-on:
            - 'self.hosted'
            - 'linux'
          size: 4x
          caches:
            - node          
          script:
            - npm install
            - npm install -g @angular/cli
            - npm run build-stage --progress=false
          artifacts:
            - dist/**
      - step:
          runs-on:
            - 'self.hosted'
            - 'linux'
          size: 4x
          name: Deploy on AWS S3 and CloudFront invalidation
          deployment: staging
          script:
            - pipe: atlassian/aws-s3-deploy:1.1.0
              variables:
                AWS_ACCESS_KEY_ID: $AWS_ACCESS_KEY_ID
                AWS_SECRET_ACCESS_KEY: $AWS_SECRET_ACCESS_KEY               
                S3_BUCKET: 'xxxxxxx/dist'
                LOCAL_PATH: 'dist'
                DELETE_FLAG: 'true'
                DEBUG: 'true'
            - pipe: atlassian/aws-cloudfront-invalidate:0.6.0
              variables:
                AWS_ACCESS_KEY_ID: $AWS_ACCESS_KEY_ID
                AWS_SECRET_ACCESS_KEY: $AWS_SECRET_ACCESS_KEY                
                DISTRIBUTION_ID: "xxxxxxx"
                PATHS: "/*"

definitions:
  caches:
    npm: $HOME/.npm
    cypress: $HOME/.cache/Cypress

and this is the initial part of my package.json:

{
  "name": "myApp",
  "version": "22.01.36"
}

I'd like the final version shown to the user is 22.01.36bxx (where xx is the Bitbucket build number). I think I just need a replacement in the package.json file. Do you have some suggestion to accomplish the "mission"?

Mr-IDE · Accepted Answer · 2022-02-08T11:15:28.130

The Bitbucket Pipelines yml file is just running Bash/shell commands on a Linux Docker machine. So you can use the normal Bash commands, like sed and perl, to do a "find and replace" inside a JSON text file.

1. Generate the correct Regular Expression

We need to write an expression that will search the text in a file for "version": "dd.dd.dd" and replace it with "version": "dd.dd.ddb123" , where "d" is a digit from 0-9.

Use https://regex101.com to write and test a regex that does this. Here's a working expression and demo and an explanation: https://regex101.com/r/sRviUF/2

Regular Expression: ("version".*:.*"\d.*(?="))
Substitution: ${1}b123

Explanation:

( and ) = Capture the found text as group 1, to use it as a substitution/replacement later
"version".*:.*" = Look for the string "version":" with 0 or more spaces allowed before and after the colon :
\d.*(?=") = Look for a single digit 0-9, then any characters. Then use a Positive Lookahead (?=") to stop the capture before the next speech mark character "
${1}b123 = Replace with the captured group 1, then add a "b" then add the numbers 123.

2. Write a Bash command to run the Regular Expression on a file, to search and replace

Test and practice on a Linux or MacOS Terminal, or use the Linux Bash Shell on Windows 10, or use an online Linux Terminal.

You will discover that the sed command cannot handle regexes with positive lookahead or negative lookahead, so we have to use perl instead. We also need to simulate the BITBUCKET_BUILD_NUMBER environment variable that is available on the Docker machine when the Pipeline is running.

# Simulate the Bitbucket Pipelines build number in our local Terminal
export BITBUCKET_BUILD_NUMBER=123

# Create a JSON text file to test with
# https://stackoverflow.com/questions/4662938/create-text-file-and-fill-it-using-bash

cat > package.json <<EOF
{
  "name": "myApp",
  "version": "22.01.36"
}
EOF

# Copy the file to a backup
cp package.json copy-package.json

# Run the regex using Perl with inline replace. Use a fixed string "123" for the build number
# https://stackoverflow.com/questions/12176026/whats-wrong-with-my-lookahead-regex-in-gnu-sed
perl -pi -e 's/("version".*:.*"\d.*(?="))/${1}b123/g' package.json

# Restore from the backup and test again
cp copy-package.json package.json

# Run the regex using Perl with inline replace. Use an environment variable for the build number
# https://unix.stackexchange.com/questions/581295/using-bash-variable-in-perl-command-in-bash
perl -pi -e 's/("version".*:.*"\d.*(?="))/${1}b$ENV{BITBUCKET_BUILD_NUMBER}/g' package.json

# Verify the file is changed correctly
cat package.json

{
  "name": "myApp",
  "version": "22.01.36b123"
}

3. Add the Bash commands to the YML script

pipelines:
  branches:
    master:
      - step:
          ...
          script:
            - npm install
            - npm install -g @angular/cli
            - npm run build-stage --progress=false

            # Get the path of the first 'package.json' which is closest to the root folder
            # https://stackoverflow.com/questions/5917576/sort-a-text-file-by-line-length-including-spaces
            - JSON_FILE_PATH=$(find "$(pwd -P)" -name package.json | awk '{ print length, $0 }' | sort -n -s | cut -d" " -f2- | head -1)
            - echo "BITBUCKET_BUILD_NUMBER = $BITBUCKET_BUILD_NUMBER"
            - echo "JSON_FILE_PATH = $JSON_FILE_PATH"

            # Debug: Print the original package.json
            - cat "$JSON_FILE_PATH"

            # Delete any existing 'bNNN' in the version string of package.json
            - perl -pi -e 's/b\d*"/"/g' "$JSON_FILE_PATH"

            # Modify the package.json to insert the $BITBUCKET_BUILD_NUMBER into the version string
            - perl -pi -e 's/("version".*:.*"\d.*(?="))/${1}b$ENV{BITBUCKET_BUILD_NUMBER}/g' "$JSON_FILE_PATH"

            # Debug: Print the modified package.json
            - cat "$JSON_FILE_PATH"

            # Possibly needed - Commit the modified package.json to Git
            - git commit -am "Add the BITBUCKET_BUILD_NUMBER into the 'version' in package.json [skip ci]"
            - git push

thanks for the detailed explanation. I'm trying to test your solution. Unfortunately the perl instruction throws an exception in the pipeline: bash: /usr/bin/perl: Argument list too long. it found all the paths where there is a package.json in all libraries. — drenda, Feb 04 '22 at 20:34
Maybe there are too many files called `package.json` in your project? Consider specifying a fixed path instead of using `JSON_FILE_PATH` which could find multiple. Or perhaps the `package.json` file is too big with too much text? Some links that may help: https://unix.stackexchange.com/questions/284170/replace-a-long-string-with-the-sed-command-argument-list-too-long-error#284172 and https://unix.stackexchange.com/questions/451486/how-to-cause-argument-list-too-long-error#451487 — Mr-IDE, Feb 04 '22 at 21:11
Please remember to add [skip ci] in the commit message to avoid infinite loop in the pipeline — drenda, Feb 04 '22 at 22:29
Yes you are right. Good point. It's now edited. I've also made `JSON_FILE_PATH` return only 1 `package.json` path, which is closest to the root folder. Probably there is a better way to do it. — Mr-IDE, Feb 05 '22 at 11:18
I realized the perl script doesn't work after the first iteration. In fact the first time I've the build number like 21.01.01, then your script add the build number at the end. The final versione will be "21.01.01b23" for example, and this file is committed. When the braches are syncronized, the script will find "21.01.01b23" version and it will add again the build number, so the final version will be "21.01.01b23b24". It should remove the previous build number if set. — drenda, Feb 07 '22 at 16:57
Probably the easiest way is to run another "find and replace" command to first delete any existing `"bNN"` in the version string. Something like this: `perl -pi -e 's/b\d*"/"/g' package.json`. I've added it into the answer. — Mr-IDE, Feb 08 '22 at 11:15
Watch out, that regexp will remove all b also from dependencies defined in the same file. That breaks everything ;-) — drenda, Feb 08 '22 at 14:21

Bitbucket Append build number to package.json during pipeline execution

1 Answers1

1. Generate the correct Regular Expression

2. Write a Bash command to run the Regular Expression on a file, to search and replace

3. Add the Bash commands to the YML script