Pagination

Retrieve large result sets efficiently using next token or timestamp-based pagination.

When result sets exceed the page limit, responses include pagination information to retrieve additional results.

Pagination Methods

Pulsedive supports two pagination approaches:

Method	Best For	How It Works
Next Token	Most use cases	Use the next value from responses to get subsequent pages.
Timestamp-Based	Time-based filtering	Use the added_after parameter with TAXII date headers.

Key Parameters

Use these parameters to control pagination:

• limit: Number of objects to return per page. Maximum server limit is 1000.
• next: Pagination token from the previous response. Used with next token pagination.
• added_after: Timestamp filter for time-based pagination.

Responses include a more field (true/false) indicating whether additional results are available. When more is true, use the next token or adjust added_after to continue retrieving objects.

Next Token Pagination

We recommend using the next value returned in responses to retrieve subsequent pages of results.

1. Make initial request with optional limit parameter.
2. Check if more is set to true in the response.
3. Use the next value returned in the response in subsequent requests to continue fetching STIX objects as part of the same result set.
4. Repeat until more is false.

Example

This sample request walks you through a typical implementation.

Initial Request

Make an initial request:

curl "https://pulsedive.com/taxii2/api/collections/a5cffbfe-c0ff-4842-a235-cb3a7a040a37/objects/ \
    ?key=<YOUR_API_KEY> \
    &accept=application/taxii+json;version=2.1 \
    &limit=500 \
    &pretty=1"

Sample requests won’t work without a valid API key. Get one by signing up for a free account.

Response

Receive a response containing more = true and the next parameter:

{
  "more": true,
  "next": "2024-01-15 10:30:45|12345",
  "objects": [
    // ... 500 STIX objects
  ]
}

Subsequent Request

Use the next value to fetch more STIX objects:

curl "https://pulsedive.com/taxii2/api/collections/a5cffbfe-c0ff-4842-a235-cb3a7a040a37/objects/ \
    ?key=<YOUR_API_KEY> \
    &accept=application/taxii+json;version=2.1 \
    &next=2024-01-15%2010:30:45|12345 \
    &limit=500 \
    &pretty=1"

Timestamp-Based Pagination

Use the added_after parameter to fetch objects added after a specific timestamp. This method is useful for incremental updates or time-based filtering.

Pulsedive includes TAXII date headers in responses to help you track which timeframes you’ve retrieved.

TAXII Date Headers

Pulsedive includes these headers in responses:

Content-Type: application/taxii+json;version=2.1
X-TAXII-Date-Added-First: 2024-01-15T10:30:45.000Z
X-TAXII-Date-Added-Last: 2024-01-15T11:45:30.000Z

These headers indicate the timestamp range of objects in the response:

• X-TAXII-Date-Added-First: created timestamp of the earliest object in this response.
• X-TAXII-Date-Added-Last: created timestamp of the latest object in this response.

Use X-TAXII-Date-Added-Last as your added_after value for the next request to continue pagination.

Example

This sample request uses the added_after filter.

Initial Request

Make an initial request:

curl "https://pulsedive.com/taxii2/api/collections/a5cffbfe-c0ff-4842-a235-cb3a7a040a37/objects/ \
    ?key=<YOUR_API_KEY> \
    &accept=application/taxii+json;version=2.1 \
    &limit=500 \
    &pretty=1"

Sample requests won’t work without a valid API key. Get one by signing up for a free account.

Receive Response

These headers are included in the response:

Content-Type: application/taxii+json;version=2.1
X-TAXII-Date-Added-First: 2024-01-15T10:30:45.000Z
X-TAXII-Date-Added-Last: 2024-01-15T11:45:30.000Z

Subsequent Request Using Last Timestamp

Use the value of the X-TAXII-Date-Added-Last header to get the next page:

curl "https://pulsedive.com/taxii2/api/collections/a5cffbfe-c0ff-4842-a235-cb3a7a040a37/objects/ \
    ?key=<YOUR_API_KEY> \
    &accept=application/taxii+json;version=2.1 \
    &added_after=2024-01-15T11:45:30.000Z \
    &limit=500 \
    &pretty=1"

You can also use the value of the created timestamp of the last STIX object in the previous response or a timestamp that you specify.

Implementation Tips

Use these strategies to handle pagination effectively.

Error Handling

• Check for more: false to detect the end of results
• Handle network timeouts and retry failed requests
• Validate pagination tokens before using them

Performance

• Use appropriate limit values (100-1000) based on your needs
• Consider caching results to avoid re-fetching the same data
• Process objects as you receive them rather than storing everything in memory