Rest to read more than 5000 items from document library

Question

Below is my code to read items from a document library.

var url = _spPageContextInfo.webAbsoluteUrl + "/_api/web/lists/getbytitle('DocumentList')/items?$select=customerID";
$.ajax({  
    url: url,  
    method: "GET",  
    headers: {  
        "Accept": "application/json; odata=verbose"  
    },  
    success: function(data) {  
        $.each(data.d.results, function(index, item) {
            arrayCustomerID[index] = item.customerID;
        });  
    },  
    error: function(data) {  
        console.log("error");  
    }  
});

This work for the normal case but what should I do so that it works if I have more than 5000 items in the document library.

please refer this answer. this will help you in this scenario - https://sharepoint.stackexchange.com/questions/74777/list-api-get-all-items-limited-to-100-rows — Rohit Waghela, Jun 05 '17 at 13:58

Rohit Waghela · Accepted Answer · 2017-06-07T12:43:25.393

14

Please refer below code :

Here we are doing recursive call to the GetListItems() function. This will fetch data in the bunch of 1000 items and concat it to response variable.

 var url = _spPageContextInfo.webAbsoluteUrl + "/_api/web/lists/getbytitle('DocumentList')/items?$select=customerID&$top=1000";
    var response = response || [];  // this variable is used for storing list items
    function GetListItems(){
        return $.ajax({
            url: url,  
            method: "GET",  
            headers: {  
                "Accept": "application/json; odata=verbose"  
            },
            success: function(data){
                response = response.concat(data.d.results);
                if (data.d.__next) {
                    url = data.d.__next;
                    GetListItems();
                }
                $.each(response, function(index, item) {
                    arrayCustomerID[index] = item.customerID;
                });
            },
            error: function(error){
            }
        });
    }

edited Jun 07 '17 at 12:43

answered Jun 07 '17 at 10:09

Rohit Waghela

2,910
2
16
31

so if (data.d.__next) will enter loop only if item number is 1001? – KumarV Jun 07 '17 at 12:27
1

@KumarV data.d.__next contains the url to get the next items. In case we don't have more items left to fetch, it will not call GetListItems() function. – Rohit Waghela Jun 07 '17 at 12:29
that mean I have to write my code inside this if loop,right? – KumarV Jun 07 '17 at 12:38
You can write your code below the if block ...please see the updated the answer... – Rohit Waghela Jun 07 '17 at 12:42
I have tried this code, but it seems not be working. Response is not getting filled. Any ideas? – KevinvG Jul 27 '17 at 10:36
@KevinvG Are you getting any error message? – Rohit Waghela Jul 28 '17 at 05:12
1

@RohitWaghela it is working now. I made a mistake myself. So above code worked, thanks. – KevinvG Jul 31 '17 at 06:32
@KevinvG Great.. :) – Rohit Waghela Jul 31 '17 at 06:46
3

If we apply some filter like _spPageContextInfo.webAbsoluteUrl + "/_api/web/lists/getbytitle('DocumentList')/items?$filter= customerID eq 'someValue'&$top=1000" then will it work? – Harsh Dec 13 '17 at 08:49
$.each(response, function(index, item) .. gets executed each time the rest call happens within GetListItems(). This isn't ideal. – entryton Sep 17 '19 at 06:11

score 1 · Answer 2 · answered Dec 17 '18 at 12:36

THIS IS WORKING !!! SOLVED

If your list or library is having more than 5000 items, then you need to perform a repeated ajax call function filtering 5000 items each time. But to know how many times to call the ajax function, we must do in Dynamic way.

First call a ajax function with parameters "/_api/Web/Lists/GetByTitle(ListName)/Items?$orderby=Id desc&$top=1". Now you will get the latest added "Id".
Now Divide the Id returned by the ajax by 5000. (Id/5000) So that you will get a result of how many times you need to perform the ajax.
Now you can perform the ajax function repeatedly be filtering every 5000 items, with filter like, $filter=Id ge 0 and Id le 5000, $filter=Id ge 5000 and Id le 10000, ........ , etc.
You can have a foreach loop to perform the ajax repeatedly with a dynamic filter, $filter=Id ge (5000*LoopValue) and Id le (5000*(LoopValue+1)).

Also make sure, the ajax to have async:true, to avoid performance issue. You can store the returned data in array and perform further actions, so that you could store more than 5000 data to perform your functions.

Since, there is a threshold limit for there kinda issues, we could use this kind of alternatives to make it possible.

While this will work, please note that MAX(Id) != COUNT(list items) as soon as some list item is deleted, so this is a bit hacky. — Jussi Palo, Jul 30 '20 at 07:17
Wouldn't a better way be to start by getting Gt 0 and then finding the max id returned and then get the next batch with id greater than that?? That way, you call them until the results are empty; right? — ChiefTwoPencils, Jan 27 '21 at 02:53

Rest to read more than 5000 items from document library

2 Answers2

THIS IS WORKING !!! SOLVED

Linked

Related