Pagination Examples

This repository contains a set of examples that demonstrate how to get paginated data from Webflow with a focus on how to get all data, one page at a time. The examples here will be built with:

• JavaScript, using the JavaScript SDK
• Python, using the Python SDK
• JavaScript, using the REST APIs directly
• Python, using the REST APIs directly

Each of the demos will make use of a Site Token for authentication. The token will require the sites:read and pages:read scopes. All code samples will read from an environment variable named WEBFLOW_SITE_TOKEN.

Pagination will be tested against the List Pages API for the first site returned by the List Sites API. For my testing purposes, I ensured my site had multiple pages.

The techniques demonstrated here could be used against any of the APIs that support pagination. Each of these APIs allow you to specify a limit (how many records to return) and offset (where to begin fetching records). While the typical maximum value of limit is 100, the code here will use a much smaller number to illustrate the paging process of fetching data until complete. In a real world application, you most likely always want to use the maximum value for limit to reduce the total number of calls required.

Each script is as simple as possible, and could be wrapped in a function. I made use of simple while loops instead of recursion. Feel free to modify to your tastes.

JavaScript SDK

The Node SDK example can be found in the js-sdk folder. Given a variable, site, that points to a site object, here's how all pages were fetched:

let hasMore = true;
let offset = 0;
let limit = 2;

const pages = [];
while(hasMore) {
    const pageReq = await webflow.pages.list(site.id, { limit, offset });
    console.log(`Pages fetched: ${pageReq.pages.length}, total is ${pageReq.pagination.total}, offset is ${pageReq.pagination.offset}`);
    hasMore = pageReq.pagination.total > offset + limit;
    pages.push(...pageReq.pages);
    offset += limit;
}

console.log(`Total pages fetched: ${pages.length}`);

Basically, do a request against the Pages API and look at the pagination part of the result. If more values exist (based on our current offset and limit), then we keep fetching via webflow.pages.list. In each loop, the data is appended to a simple array.

JavaScript

The JavaScript, non-SDK one simply replaces the SDK call with a HTTP call made using fetch. The response from the SDK is the exact same as that with the direct API call.

let hasMore = true;
let offset = 0;
let limit = 2;

const pages = [];
while(hasMore) {
    const pageReq = await fetch(`https://api.webflow.com/v2/sites/${site.id}/pages?limit=${limit}&offset=${offset}`, {
        headers: {
            'Authorization': `Bearer ${accessToken}`
        }
    });
    const pageData = await pageReq.json();
    console.log(`Pages fetched: ${pageData.pages.length}, total is ${pageData.pagination.total}, offset is ${pageData.pagination.offset}`);
    hasMore = pageData.pagination.total > offset + limit;
    pages.push(...pageData.pages);
    offset += limit;
}

console.log(`Total pages fetched: ${pages.length}`);

Python SDK

Please note the Python demo is using version 2.0.0b2. After fetching the first site available via the provided key, this code fetches all the pages:

has_more = True
offset = 0
limit = 2

while has_more:
    pageReq = client.pages.list(site_id = site.id, limit = limit, offset= offset)

    for page in pageReq.pages:
        pages.append(page)

    print(f"Pages fetched: {len(pageReq.pages)}, total is {pageReq.pagination.total}, offset is {pageReq.pagination.offset}")

    has_more = pageReq.pagination.total > offset + limit
    offset += limit

print(f"Total pages fetched: {len(pages)}")

Python

As with the JavaScript SDK/non-SDK versions, the only changes here are hitting the API directly instead of the SDK:

has_more = True
offset = 0
limit = 2

while has_more:

    pageReq = requests.get(f"https://api.webflow.com/v2/sites/{site['id']}/pages", headers = {
        "Authorization": f"Bearer {access_token}",
    }, params = {
        "limit": limit,
        "offset": offset
    }).json()

    for page in pageReq["pages"]:
        pages.append(page)

    print(f"Pages fetched: {len(pageReq['pages'])}, total is {pageReq['pagination']['total']}, offset is {pageReq['pagination']['offset']}")

    has_more = pageReq['pagination']['total'] > offset + limit
    offset += limit

print(f"Total pages fetched: {len(pages)}")

Other Examples

These examples are external and are not officially supported by Webflow.

• "Webflow API pagination in Make.com", blog post by BORING