Skip to content

Commit

Permalink
add:添加资源同步进度条
Browse files Browse the repository at this point in the history
  • Loading branch information
GentleCP committed Apr 17, 2020
1 parent c2308e5 commit 84aa020
Show file tree
Hide file tree
Showing 6 changed files with 46 additions and 4 deletions.
5 changes: 4 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,9 +45,12 @@

# 2. 版本号

1.5.1
1.6.0

## 2.1 更新内容
- [1.6.0]
> 文件同步添加了进度条,方便查看文件同步信息,但请注意,若下载过程中未完成中断任务需将下载一半的文件删除,否则不会更新
![](img/1.6.0.png)
- [1.5.1]
> 修复下载全部课程资源时没有打开文件目录的选项问题,当无更新的时候自动退出应用
Expand Down
9 changes: 6 additions & 3 deletions core/source.py
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,8 @@

from bs4 import BeautifulSoup

from core.utils import download_file

sys.setrecursionlimit(100000)


Expand Down Expand Up @@ -211,9 +213,10 @@ def _download_one(self, course_info, source_info):
if not os.path.isfile(file_path):
# 只下载没有的文件,若存在不下载,节省流量
self._logger.info("正在下载:{}".format(source_info['name']))
content = self._S.get(source_info['url']).content
with open(file_path, 'wb') as f:
f.write(content)
# content = self._S.get(source_info['url']).content
# with open(file_path, 'wb') as f:
# f.write(content)
download_file(url=source_info['url'], session=self._S, file_path=file_path)
self._update_sources.append("[{}:{}]".format(course_info["name"], source_info['name'])) # 记录更新的课程数据

def _download_course(self, course_info):
Expand Down
34 changes: 34 additions & 0 deletions core/utils.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,9 @@
# @Time : 2019/11/28/028 14:11
# @WebSite : https://www.gentlecp.com
import requests
import os
import logging
from tqdm import tqdm
from urllib.parse import urlparse

def util_login(stuid,password):
Expand Down Expand Up @@ -33,3 +36,34 @@ def util_login(stuid,password):
}


def download_file(url, session=None, file_path='未命名文件', overwrite = False):
'''
根据指定url下载文件
:param url:
:param session: 传入的会话参数,有的需要登录才能下载
:param file_path: 文件存储路径,默认为当前目录下,存储文件为未命名文件
:param overwrite: 是否覆盖同名文件,默认否
:return: 正确下载返回True,否则False
'''
if session:
res = session.get(url, stream = True)
else:
res = requests.get(url,stream=True)
file_size = int(res.headers['content-length'])
chunk_size = 1024
if res.status_code == 200:
if not overwrite and os.path.exists(file_path):
return True
else:
progress_bar = tqdm(
total=file_size,initial=0,unit='B',unit_scale=True,
)
with open(file_path,'wb') as f:
for data in res.iter_content(chunk_size=chunk_size):
if data:
f.write(data)
progress_bar.update(chunk_size)
progress_bar.close()
else:
logging.error('download fail.')
return False
1 change: 1 addition & 0 deletions environment.yml
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,7 @@ dependencies:
- prettytable==0.7.2
- requests==2.22.0
- soupsieve==1.9.5
- tqdm==4.45.0
- urllib3==1.25.7
prefix: /Users/dongchaopeng/miniconda3/envs/UCASHelper

Binary file added img/1.6.0.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
1 change: 1 addition & 0 deletions requirements.txt
Original file line number Diff line number Diff line change
Expand Up @@ -5,4 +5,5 @@ idna==2.8
prettytable==0.7.2
requests==2.22.0
soupsieve==1.9.5
tqdm==4.45.0
urllib3==1.25.7

0 comments on commit 84aa020

Please sign in to comment.