As documented in this thread, I’ve been working on a long-running, periodic job to pull data from Okta using the Java SDK. In order to ensure completeness and minimize duplicates across restarts of the job, I am currently storing the publish time of the last seen LogEvent as a checkpoint. However, when my job restarts from the checkpoint, I get duplicate copies of the event from which I got the checkpoint time and any other events occurring in the same second.
I see that the SDK is using the REST API’s
next link and I was even able to get and use the value of the
next link by doing something like this:
// Get the NEXT link LogEventList logs = client.getLogs(); String nextPageHref = ((DefaultLogEventList) logs).getString("nextPage"); // Some time later... get the LogEventList for the previously obtained NEXT link client.getDataStore().getResource(nextPageHref, LogEventList.class);
However, I’m reluctant to do this in code that matters because it’s digging into internals of the SDK that I assume the developers didn’t intend to expose and a future SDK update could break my code by - for example - changing the name of the property where the
next link is stored.
I’d like to request that features be added to the Okta Java SDK that make it easier to do checkpointing by vending a checkpoint-able value and accepting that same value in query functionality.