Okta sdk 3.0 - how are you supposed to paginate?

Questions
1

previously in the python sdk (prior to version 3.0 overhaul) - the response would cast into OktaAPIResponse - which has a has_next() and, next() for pagination,
currently the responses are returned as ApiResponse.

how are you supposed to paginate given the response?

2

Hi @orishalhon. I reached out to our Python SDK team and they provided the following information:

Pagination is supported in v3.0, though currently, it requires manual handling of the cursor rather than an automatic iterator.
You can achieve this using the limit and after parameters available in the operation functions. Here is the workflow:

  1. Limit: Use the limit parameter to define the page size.
  2. Cursor: The response headers contain a Link attribute (e.g., <https://.../users?after=abc>; rel="next").
  3. Next Page: Extract the after value from that URL and pass it to the after parameter in your next function call to fetch the subsequent page.

Here is a quick example of how to handle this:

import urllib.parse

# 1. First request
apps, resp, err = await client.list_applications(limit=5)

# 2. Extract 'after' cursor from Link header
headers = resp.headers
def extract_next_cursor(headers: Dict[str, Any]) -> Optional[str]:
    try:
        # Try both capitalized and lowercase 'link' headers
        link_header = headers.get('Link') or headers.get('link')

        if not link_header:
            return None

        # Parse the Link header to find the 'next' link
        # Format: <URL>; rel="next", <URL>; rel="self"
        links = link_header.split(',')

        for link in links:
            # Check if this is the 'next' link
            if 'rel="next"' in link or "rel='next'" in link:
                # Extract URL from <URL>
                url_part = link.split(';')[0].strip()
                url = url_part.strip('<>').strip()

                # Extract 'after' parameter from URL
                parsed = urlparse(url)
                query_params = parse_qs(parsed.query)

                if 'after' in query_params:
                    # Return the first value if it's a list
                    after_value = query_params['after']
                    if isinstance(after_value, list) and len(after_value) > 0:
                        return after_value[0]
                    return str(after_value)

        return None

    except Exception:
        # Silently return None on parsing errors
        return None
after_cursor = extract_next_cursor(headers)
# 3. Fetch next page
if after_cursor:
    next_apps, next_resp, next_err = await client.list_applications(after=after_cursor, limit=5)

We are planning to introduce a helper feature in a future release to handle this pagination more gracefully (automatic iteration). For now, please use the approach above.

3

andrea, thanks for your response (and hi)

your example still needs to loop. i’d recommend using yield (see OktaApiResponse is not used, therefore has_next is not available. · Issue #478 · okta/okta-sdk-python · GitHub)

also i know requests will parse the link header for you. i’m guessing most Python libraries do it.

lastly, headers are case-insensitive (should be lowercase since Okta uses http/2)

1 Like
4

here’s a more complete example (add the extract_next_cursor from above)

import okta.client
import asyncio
from urllib.parse import urlparse, parse_qs

async def main():
    async with okta.client.Client() as client:
        apps, resp, err = await client.list_applications(limit=5)
        while True:
            for app in apps:
                print(app.id, app.label)

            after = extract_next_cursor(resp.headers)
            if after:
                apps, resp, err = await client.list_applications(after=after, limit=5)
            else:
                break

asyncio.run(main())