How to get more than 1000 actions from a Trello board?

Question

I want to run a simple script which gets in JSON form all the cards, etc. from Trello which data & URL looks like this: https://trello.com/b/Vqrkz3KO.json. I want to automate the process of exporting a board.

But Trello won't let me do that, so I have to do it from the API, like this:

`/1/boards/${board.id}`, {fields: "all",actions: "all",action_fields: "all",activities_limit: 1000,cards: "all",card_fields: "all",card_attachments: true,labels: "all",lists: "all",list_fields: "all",members: "all",member_fields: "all",checklists: "all",checklist_fields: "all", organization: false

It does what I want it to do, but the boards which I want to get the cards from have MORE than 1000 activities. So How do I get all the activities? It only fetches 1000 activities/cards. Also, I cant just fetch the URL above because for some reason you cant use the api to fetch it as per this question. This question doesn't answer my question, I want to go past 1000 cards, but if you do, you get an error saying you can't.

Answer 1

EDIT 2: Finally got it, I used the API & a ID generator for HTTPS requests and used the provided &before= parameter. The &before= parameter uses dates as the arguments, so I had to grab the last action from one request, get the date from that, and provide it for the &before parameter. Then for every array element that had 1000 elements, I popped the last element because I'd end up with duplicate actions.

Now, I ended up with actions that look like this: [[actions],[actions],[actions],[actions]] and so on, so I used Merge/flatten an array of arrays 's answer to make it all [actions]. Then I used bracket notation object["key"] = value to set / replace the actions with the actions from my HTTPS requests, and it came out to a VERY large file, and it took quite a while to produce this file, it came out to around 99.5 MB.

this is my whole index.js test file:

const https = require('https');
const fs = require('fs');
var boardinfo = "";
https.get({
    hostname: 'trello.com',
    path: `/b/....json`,
    headers: {'User-Agent': `${Math.random().toString(16).substring(2,16)}`}
}, (r) => {
    var data = "";
    r.on('data', (d) => {
        data+=d;
    })
    r.on('close', () => {
        boardinfo = JSON.parse(data);
    });
})

var actions = [];

(function untilDeath(beforeval) {
https.get({
    hostname: 'api.trello.com',
    path: `/1/boards/.../actions?limit=1000${beforeval ? `&before=${beforeval}` : ``}`,
    headers: {'User-Agent': `${Math.random().toString(16).substring(2,16)}`}
}, (r) => {
    var cmpdta = "";
    r.on('data', (d) => {
        cmpdta+=d;
    })
    r.on('close', () => {
        cmpdta = JSON.parse(cmpdta);
        if(cmpdta.length < 1000) {
            if(cmpdta.length) actions.push(cmpdta);
            return makeFile(info, [].concat.apply([], actions), fileName);
        } else
        untilDeath(cmpdta[999].date);
        cmpdta.pop();
        actions.push(cmpdta);
    });

    r.on('error', () => {
        throw new Error('Error');
    });
});
})();

function makeFile(trelloBoard, actions) {
    trelloBoard["actions"] = actions;
    fs.createWriteStream('./full-board.json');
    fs.writeFile(`./full-board.json`, JSON.stringify(trelloBoard, null, `\t`), (c) => {
        if(c) console.log(c);
    });
}

EDIT: Disappointingly, this also only fetches 1000 actions, even with saving the JSON file manually, it still gives 1000 actions.

I easily resolved this with a HTTPS User-Agent header.

const https = require('https');
https.get({
    hostname: 'trello.com',
    path: '/b/....json',
    headers: {'User-Agent': 'some-random-user-agent'}
}, (r) => {
    var str = "";
    r.on('data', (d) => {str+=d});
    r.on('close', () => {console.log(str)})
})