1

I try to use Bitbucket's pipeline feature for a LaTeX git repository. I just want to build my .tex file and store the .pdf artifact to the repository download folder. I found some helpful guides here and a similar answer at SO.

This is my pipeline.yml

options:
  docker: true

image: kaspersoerensen/latex-docker

pipelines:
  default:
    - step:
        script:
          - pdflatex --shell-escape TEST.tex # Build once
          #- bibtex TEST # Build bibtex
          - pdflatex -shell-escape TEST.tex # Build again
          - pdflatex -shell-escape TEST.tex # And last time
          - curl -X POST --user "${BB_AUTH_STRING}" "https://api.bitbucket.org/2.0/repositories/${BITBUCKET_REPO_OWNER}/${BITBUCKET_REPO_SLUG}/downloads" --form files=@"TEST.pdf"

To setup curl I used this official guideline from Atlassian.

Everything seems to be okay and the build is successful without any errors.

Problem: My repository download folder does not contain any artifacts.

EDIT

Build output:

Build setup
4s
pdflatex --shell-escape TEST.tex
1s
pdflatex -shell-escape TEST.tex
1s
pdflatex -shell-escape TEST.tex
1s
curl -X POST --user "${BB_AUTH_STRING}" "https://api.bitbucket.org/2.0/repositories/${BITBUCKET_REPO_OWNER}/${BITBUCKET_REPO_SLUG}/downloads" --form files=@"TEST.pdf"
<1s
Build teardown

Docker output:

time="2019-12-20T14:56:23.511294820Z" level=warning msg="could not change group /var/run/docker.sock to docker: group docker not found"
time="2019-12-20T14:56:23.511451739Z" level=warning msg="[!] DON'T BIND ON ANY IP ADDRESS WITHOUT setting --tlsverify IF YOU DON'T KNOW WHAT YOU'RE DOING [!]"
time="2019-12-20T14:56:23.544238141Z" level=warning msg="failed to load plugin io.containerd.snapshotter.v1.btrfs" error="path /var/lib/docker/165536.165536/containerd/daemon/io.containerd.snapshotter.v1.btrfs must be a btrfs filesystem to be used with the btrfs snapshotter" 
time="2019-12-20T14:56:23.562813661Z" level=warning msg="failed to load plugin io.containerd.snapshotter.v1.aufs" error="modprobe aufs failed: "ip: can't find device 'aufs'\nmodprobe: can't change directory to '/lib/modules': No such file or directory\n": exit status 1" 
time="2019-12-20T14:56:23.563473734Z" level=warning msg="failed to load plugin io.containerd.snapshotter.v1.zfs" error="path /var/lib/docker/165536.165536/containerd/daemon/io.containerd.snapshotter.v1.zfs must be a zfs filesystem to be used with the zfs snapshotter" 
time="2019-12-20T14:56:23.563610398Z" level=warning msg="could not use snapshotter btrfs in metadata plugin" error="path /var/lib/docker/165536.165536/containerd/daemon/io.containerd.snapshotter.v1.btrfs must be a btrfs filesystem to be used with the btrfs snapshotter" 
time="2019-12-20T14:56:23.563626718Z" level=warning msg="could not use snapshotter aufs in metadata plugin" error="modprobe aufs failed: "ip: can't find device 'aufs'\nmodprobe: can't change directory to '/lib/modules': No such file or directory\n": exit status 1" 
time="2019-12-20T14:56:23.563636848Z" level=warning msg="could not use snapshotter zfs in metadata plugin" error="path /var/lib/docker/165536.165536/containerd/daemon/io.containerd.snapshotter.v1.zfs must be a zfs filesystem to be used with the zfs snapshotter" 
time="2019-12-20T14:56:23.695604523Z" level=warning msg="Running modprobe bridge br_netfilter failed with message: ip: can't find device 'bridge'\nbridge                167936  1 br_netfilter\nstp                    16384  1 bridge\nllc                    16384  2 bridge,stp\nip: can't find device 'br_netfilter'\nbr_netfilter           24576  0 \nbridge                167936  1 br_netfilter\nmodprobe: can't change directory to '/lib/modules': No such file or directory\n, error: exit status 1"
time="2019-12-20T14:56:23.706219675Z" level=warning msg="Running modprobe nf_nat failed with message: `ip: can't find device 'nf_nat'\nnf_nat_ipv6            16384  1 ip6table_nat\nnf_nat_ipv4            16384  2 ipt_MASQUERADE,iptable_nat\nnf_nat                 32768  3 nf_nat_ipv6,xt_nat,nf_nat_ipv4\nnf_conntrack          139264  8 nf_nat_ipv6,nf_conntrack_netlink,xt_nat,xt_conntrack,ip_vs,ipt_MASQUERADE,nf_nat_ipv4,nf_nat\nlibcrc32c              16384  3 ip_vs,nf_nat,nf_conntrack\nmodprobe: can't change directory to '/lib/modules': No such file or directory`, error: exit status 1"
time="2019-12-20T14:56:23.716309547Z" level=warning msg="Running modprobe xt_conntrack failed with message: `ip: can't find device 'xt_conntrack'\nxt_conntrack           16384 42 \nnf_conntrack          139264  8 nf_nat_ipv6,nf_conntrack_netlink,xt_nat,xt_conntrack,ip_vs,ipt_MASQUERADE,nf_nat_ipv4,nf_nat\nmodprobe: can't change directory to '/lib/modules': No such file or directory`, error: exit status 1"
PascalS
  • 975
  • 1
  • 16
  • 40

2 Answers2

2

I'd recommend to use the bitbucket-upload-file pipe instead of sending API requests with curl to upload files in Bitbucket. The pipe will provide better feedback if something is wrong. Here is an example how you could do that:

 options:
   docker: true

 image: kaspersoerensen/latex-docker

 pipelines:
   default:
     - step:
         name: Build the pdf
         script:
           - pdflatex --shell-escape TEST.tex
         artifacts:
           - TEST.pdf
     - step:
         name: Upload
         script:
           - pipe: atlassian/bitbucket-upload-file:0.1.2
             variables:
               BITBUCKET_USERNAME: $BITBUCKET_USERNAME
               BITBUCKET_APP_PASSWORD: $BITBUCKET_APP_PASSWORD
               FILENAME: 'TEST.pdf'

Note, that I'm using to separate steps to build and upload, which is considered a good way to structure your pipelines.

Alexander Zhukov
  • 4,357
  • 1
  • 20
  • 31
0

Thanks to everyone who spent time to answer my question.

Now I found a working solution like this:

  1. In general follow this nice written guide!
  2. Be sure that you use two factor authentification!
  3. BB_AUTH_STRING must be YOUR_USERNAME:YOUR_APP_PASSWORD (Don't use the App Password label! This was my fault)
  4. Use the following .yml code.

.

image: dme26/latex-builder:latest
pipelines: 
  default:
  - step:
    script:  
    - pdflatex TEST.tex; pdflatex TEST.tex
    - ls
    - curl -X POST "https://${BB_AUTH_STRING}@api.bitbucket.org/2.0/repositories/${BITBUCKET_REPO_OWNER}/${BITBUCKET_REPO_SLUG}/downloads" --form files=@"TEST.pdf"
PascalS
  • 975
  • 1
  • 16
  • 40