Search pagination in "next" API


#1

I’m seeing something odd in the behavior of the search API, in particular the use of the limit and offset arguments to paginate output. Here’s a first call:

https://api.next.nomics.world/search?q=BOE&limit=20&offset=0

This gives me 20 datasets OK (although, surprisingly, number 20 is Eurostat’s “fish_ld_pt”). I then try pagination: I repeat the call twice, first with limit=10, offset=0, then with limit=10, offset=10. I’d expect the first of these to produce results 1-10 of the original 20, and the second to give results 11-20. But in fact the latter two calls give exactly the same result, and it’s not equal to either 1-10 or 11-20 of the original.


#2

Let me attempt to clarify: the “not equal to either 1-10 or 11-20 of the original” part of my original posting may be irrelevant, perhaps I was assuming too much. But here’s the key point: calls to the search API with arguments

(a) BOE, limit=10, offset=0
(b) BOE, limit=10, offset=10

produce the same result, when it seems clear that they should produce disjoint subsets of the full list of matching datasets. This is not restricted to a search on “BOE”, it seems to be quite general: the “search” API in effect ignores the offset argument. This may be verified by using the (very useful) online API explorer for “search”, using any string.


#3

Hi @acottrell,

Thanks for taking time to report this.

I will look a that more in depth tomorrow, to confirm what you explain, and fix it.

Regards


#4

To be a little more precise, here are full examples of what I’m talking about.

“general” search: offset ignored: these two searches give the same result
https://api.next.nomics.world/search?q=interest%20rate&limit=10&offset=0
https://api.next.nomics.world/search?q=interest%20rate&limit=10&offset=10

Compare with:

search specific to dataset: offset works OK: results differ as expected
https://api.next.nomics.world/ECB/BLS/series?q=interest%20rate&limit=10&offset=0
https://api.next.nomics.world/ECB/BLS/series?q=interest%20rate&limit=10&offset=10


#5

Hi @acottrell

I created this issue corresponding to the bug report you made, I fixed the problem and deployed the new version in production.

Can you please confirm that it’s OK for you?

Sorry for the delay, and thanks again for all the feedback you made on DBnomics.

Regards,

Christophe


#6

Yes, confirmed OK now. Thanks!


Incorrect "num_found" in search results