How to Bypass the 5000 Folder/File Item Limit for a SharePoint Document Library Folder?

[Yes-Yes-Yes-Yes-Yes]

I recognize that existing issues and examples such as #392 and Office-365 REST Client example read_all.py tackles how to incrementally query more than 5000 items from a SharePoint list, but I found no solutions on how to similarly incrementally query more than 5000 items from a document library. Most solutions that I found online sought to resolve this issue through editing the site permissions itself such as increasing the file query limit, but I have no access to such permissions. Is there a way to get around these restrictions using existing SharePoint Python code?

---

If anyone is interested, below is the code I am using:

class SHARE_LIB:
    __context = ""

  class ShareItem:
    __name = ""
    __type = ""
    __last_modified = ""
    __relative_url = ""
  
    def __init__(self, name, type, last_modified, relative_url):
        self.__name = name
        self.__type = type
        self.__last_modified = last_modified
        self.__relative_url = relative_url
    
    def get_name(self):          return self.__name
    def get_type(self):          return self.__type
    def get_last_modified(self): return self.__last_modified
    def get_relative_url(self):  return self.__relative_url

  def __init__(self, context):
    self.__context = context
    
  def get_folder_small(self, relative_url, recursive = False):
      item_list = []
  
      try:
          folder = self.__context.web.get_folder_by_server_relative_url(relative_url)
          sub_folders = (folder.get_folders(recursive).execute_query())               # ERROR HERE
          files = folder.get_files(recursive).execute_query()                         # ERROR HERE
      except Exception as e:
          print("------------------------------------------------------")
          print(f"ERROR Class: {e.__class__.__name__}")
          print(f"ERRROR: {e}")
          
          input("Press any key to continue...")
          exit(1)
  
      for folder in sub_folders:
          item_name = folder.properties["Name"].strip()
          item_type = FileSystemObjectType.Folder
          item_last_modified = localize_sharepoint_modified(folder.properties["TimeLastModified"])
          item_url = folder.properties["ServerRelativeUrl"].strip()
      
          item = self.ShareItem(item_name, item_type, item_last_modified, item_url)
          item_list.append(item)
  
      for file in files:
          item_name = file.properties["Name"].strip()
          item_type = FileSystemObjectType.File
          item_last_modified = localize_sharepoint_modified(file.properties["TimeLastModified"])
          item_url = file.properties["ServerRelativeUrl"].strip()
  
          item = self.ShareItem(item_name, item_type, item_last_modified, item_url)
          item_list.append(item)
  
      item_list.sort(key = lambda x: x.get_name().upper())
      return item_list

ERROR when I run the code:

ERROR Class: ClientRequestException
ERRROR: ('-2147024860, Microsoft.SharePoint.SPQueryThrottledException', 'The attempted operation is prohibited because it exceeds the list view threshold.', "500 Server Error: Internal Server Error for url: https://fsrao.sharepoint.com/sites/<site_name>/_api/Web/getFolderByServerRelativeUrl('<relative_url>')?$select=Folders&$expand=Folders")

[KiranGangadhar01]

@Yes-Yes-Yes-Yes-Yes did you find any solution for the issue?

[Danxx26hub]

This is a sharepoint limitation.. once your view has more than 5K rows of data, you won’t be able to pull data without an index. You will need to go to your list settings and create an index on one or more columns. Preferably this should be done before creating a list in sharepoint.

[Justin-Yi-Cheng]

@Yes-Yes-Yes-Yes-Yes did you find any solution for the issue?

Unfortunately I was not able to resolve this issue yet...