Understanding Pagination, and Best Practices for ShipEngine APIs

Some ShipEngine API endpoints, such as the one to list labels, paginate their responses to make the results easier to manage. Pagination allows the endpoint to divide the results into discrete batches and return a single page at a time—reducing the strain on response and load times. When you call a paginated endpoint, it will return the first batch of data along with a marker that can be used to request the next set of results. 

In order for your application to consume all the data available from one of these endpoints, it’s important to understand what pagination is, why we use it in some of ShipEngine’s API endpoints, and what you need to do in order for your application to consume all the data.

What is Pagination?

Pagination is simply a way to break up the results from an API call into smaller chunks that your application can request individually. ShipEngine API uses page-based pagination, and the marker used to request the next batch of results is referred to as a page. You can think of it as a book with multiple numbered pages where each page of the book contains a portion of the book as a whole.  

So how does a book with multiple pages relate to an application consuming a paginated API? Think of a time when you’ve shopped online and flipped through pages and pages of a particular category. We’ll use this screenshot to jog your memory.

You may recall seeing something similar to the drop-down box in green in the screenshot above. This box allows you to either select a specific page from the drop-down or click the right-hand arrow to simply advance to the next page. Once you land on any page other than the first page, an arrow will also appear on the left that allows you to navigate backward. 

This is pagination. You can bet that the websites you use for online shopping are calling an API to populate their inventory. When you select a specific page or click the arrow to advance to the next page, the website makes an API request in real-time to gather data for that page. 

Another aspect of pagination is the page size. You can see an example of this in the red box in the screenshot above. In this case, there are 60 items per page. But we could change the value to 120 to see double the number of items per page. This is known as the page size. A larger page size means fewer calls to the API endpoint, but it also means twice the amount of data will be returned from a single API call. 

The Importance of Paginating API Calls

Using the example above, we would have received 7,980 products without the use of pagination (133 pages times 60 items per page). Now, imagine that you have created hundreds of thousands of labels using ShipEngine API. Without pagination, your request to list labels will return a large amount of data, some of which may not even be useful to you in that particular moment. 

Not only is this an unmanageable amount of data for your application, but it can also slow the API and decrease performance for all users. Executing a query against ShipEngine API’s database that returns hundreds of thousands of labels requires more processing time than requesting a single page of labels. 

Paginating results for endpoints that return a list or collection of items makes it easier for your application to work with the data as well as to ensure optimal performance of ShipEngine API.  

How to Implement Pagination in ShipEngine?

The endpoints that use pagination in ShipEngine API return a response similar to the sample response below. The items in the first page of data have been removed from the example for brevity. 

...
  "total": 95,
   "page": 1,
   "pages": 4,
   "links": {
       "first": {
           "href": "https://api-stage.shipengine.com/v1/labels?page=1&page_size=25"
       },
       "last": {
           "href": "https://api-stage.shipengine.com/v1/labels?page=4&page_size=25"
       },
       "prev": {},
       "next": {
           "href": "https://api-stage.shipengine.com/v1/labels?page=2&page_size=25"
       }
   }

This paginated response includes the following details:

• The total number of labels – 95 
• The page that was returned – 1 
• The total number of pages – 4 
• A link to the first page of results in links.first.href 
• A link to the last page of results in links.last.href 
• A link to the previous page of results in prev, which is empty since we are on the first page of results
• A link to the next page of results in next.href

To understand how to use pagination in ShipEngine API, let’s look at the URLs used to retrieve the first, next, previous, and last page in the example above. These URLs include two URL query parameters;

• page, and 
• page_size 

These links are convenience links to help you access those pages easily. When you call the endpoint from your application, you will use those same query parameters to indicate the page you would like to retrieve as well as the page_size. You can also eliminate the page_size parameter and it will use the default value of 25 items per page. 

Best Practice: Filtering Pagination Results 

Some ShipEngine API endpoints return a list or collection of items, and these endpoints use pagination to keep the results manageable. The total number of items is only going to continue to grow over time, so the total number of pages is also going to increase over time. At some point, the number of pages themselves may become unmanageable.

You should use URL query parameters in your request to filter the items up front and reduce the total number of pages in your response. 

For example, when you list labels, you can ask for labels for a particular warehouse or carrier or those created during a specific time period. The response will now contain only those labels that match the criteria you specified, resulting in fewer pages for your application to consume, and giving you only the labels you really care about in the first place. 

If you have never used pagination and URL query parameters before, our docs and code samples on Github will help you get started. Get a free API key to start building (and paginating) with ShipEngine.