Skip to content

Commit

Permalink
javbus: 解决爬虫被识别时返回404导致无法解析站点数据的问题
Browse files Browse the repository at this point in the history
  • Loading branch information
Yuukiy committed Nov 30, 2023
1 parent 536f4f4 commit 6f5b711
Showing 1 changed file with 0 additions and 3 deletions.
3 changes: 0 additions & 3 deletions web/javbus.py
Original file line number Diff line number Diff line change
Expand Up @@ -28,9 +28,6 @@ def parse_data(movie: MovieInfo):
"""
url = f'{base_url}/{movie.dvdid}'
resp = request_get(url, delay_raise=True)
if resp.status_code == 404:
raise MovieNotFoundError(__name__, movie.dvdid)
resp.raise_for_status()
# 疑似JavBus检测到类似爬虫的行为时会要求登录,不过发现目前不需要登录也可以从重定向前的网页中提取信息
if resp.history and resp.history[0].status_code == 302:
html = resp2html(resp.history[0])
Expand Down

0 comments on commit 6f5b711

Please sign in to comment.