Skip to content

Commit

Permalink
Updated documentation for the load function
Browse files Browse the repository at this point in the history
  • Loading branch information
yeounhak committed Dec 18, 2024
1 parent d357cb3 commit 339f196
Showing 1 changed file with 1 addition and 24 deletions.
25 changes: 1 addition & 24 deletions docs/docs/integrations/document_loaders/web_base.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -154,29 +154,6 @@
"There are reasonable limits to concurrent requests, defaulting to 2 per second. If you aren't concerned about being a good citizen, or you control the server you are scraping and don't care about load, you can change the `requests_per_second` parameter to increase the max concurrent requests. Note, while this will speed up the scraping process, but may cause the server to block you. Be careful!"
]
},
{
"cell_type": "code",
"execution_count": 9,
"id": "9f9cf30f",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Requirement already satisfied: nest_asyncio in /Users/harrisonchase/.pyenv/versions/3.9.1/envs/langchain/lib/python3.9/site-packages (1.5.6)\n"
]
}
],
"source": [
"%pip install -qU nest_asyncio\n",
"\n",
"# fixes a bug with asyncio and jupyter\n",
"import nest_asyncio\n",
"\n",
"nest_asyncio.apply()"
]
},
{
"cell_type": "code",
"execution_count": 10,
Expand All @@ -198,7 +175,7 @@
"source": [
"loader = WebBaseLoader([\"https://www.espn.com/\", \"https://google.com\"])\n",
"loader.requests_per_second = 1\n",
"docs = await loader.aload()\n",
"docs = loader.load()\n",
"docs"
]
},
Expand Down

0 comments on commit 339f196

Please sign in to comment.