Skip to content

Dev (Sourcery refactored) #18

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
sourcery-ai wants to merge 1 commit into
base: dev
Choose a base branch
from
Open

Dev (Sourcery refactored) #18

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion Schedule/demo4.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@


def greet(name):
print('Hello {}'.format(name))
print(f'Hello {name}')
Copy link
Author

@sourcery-ai sourcery-ai bot Oct 26, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Function greet refactored with the following changes:



# .tag 打标签
Expand Down
6 changes: 3 additions & 3 deletions Schedule/demo8.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,11 +11,11 @@
不过你可以通过多线程的形式来运行每个作业以解决此限制：
"""
def job1():
print("I'm running on threads %s" % threading.current_thread())
print(f"I'm running on threads {threading.current_thread()}")
Copy link
Author

@sourcery-ai sourcery-ai bot Oct 26, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Function job1 refactored with the following changes:

def job2():
print("I'm running on threads %s" % threading.current_thread())
print(f"I'm running on threads {threading.current_thread()}")
Copy link
Author

@sourcery-ai sourcery-ai bot Oct 26, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Function job2 refactored with the following changes:

def job3():
print("I'm running on threads %s" % threading.current_thread())
print(f"I'm running on threads {threading.current_thread()}")
Copy link
Author

@sourcery-ai sourcery-ai bot Oct 26, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Function job3 refactored with the following changes:


def run_threaded(job_func):
job_thread = threading.Thread(target=job_func)
Expand Down
14 changes: 6 additions & 8 deletions chrome/chrome_helper.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,9 +10,9 @@
# https://medium.com/drunk-wis/python-selenium-chrome-browser-%E8%88%87-driver-%E6%83%B1%E4%BA%BA%E7%9A%84%E7%89%88%E6%9C%AC%E7%AE%A1%E7%90%86-cbaf1d1861ce
CHROME_DRIVER_BASE_URL = "https://chromedriver.storage.googleapis.com"
CHROME_DRIVER_FOLDER = r".\chrome"
CHROME_DRIVER_MAPPING_FILE = r"{}\mapping.json".format(CHROME_DRIVER_FOLDER)
CHROME_DRIVER_EXE = r"{}\chromedriver.exe".format(CHROME_DRIVER_FOLDER)
CHROME_DRIVER_ZIP = r"{}\chromedriver_win32.zip".format(CHROME_DRIVER_FOLDER)
CHROME_DRIVER_MAPPING_FILE = f"{CHROME_DRIVER_FOLDER}\mapping.json"
CHROME_DRIVER_EXE = f"{CHROME_DRIVER_FOLDER}\chromedriver.exe"
CHROME_DRIVER_ZIP = f"{CHROME_DRIVER_FOLDER}\chromedriver_win32.zip"
Comment on lines -13 to +15
Copy link
Author

@sourcery-ai sourcery-ai bot Oct 26, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lines 13-15 refactored with the following changes:



def init_dir():
Expand All @@ -30,8 +30,7 @@ def get_chrome_driver_major_version():


def get_latest_driver_version(browser_ver):
latest_api = "{}/LATEST_RELEASE_{}".format(
CHROME_DRIVER_BASE_URL, browser_ver)
latest_api = f"{CHROME_DRIVER_BASE_URL}/LATEST_RELEASE_{browser_ver}"
Copy link
Author

@sourcery-ai sourcery-ai bot Oct 26, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Function get_latest_driver_version refactored with the following changes:

logger.info(f'latest api is {latest_api}')
resp = requests.get(latest_api)
lastest_driver_version = resp.text.strip()
Expand All @@ -40,8 +39,7 @@ def get_latest_driver_version(browser_ver):


def download_driver(driver_ver, dest_folder):
download_api = "{}/{}/chromedriver_win32.zip".format(
CHROME_DRIVER_BASE_URL, driver_ver)
download_api = f"{CHROME_DRIVER_BASE_URL}/{driver_ver}/chromedriver_win32.zip"
Comment on lines -43 to +42
Copy link
Author

@sourcery-ai sourcery-ai bot Oct 26, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Function download_driver refactored with the following changes:

logger.info(f'download api is {download_api}')
dest_path = os.path.join(dest_folder, os.path.basename(download_api))
resp = requests.get(download_api, stream=True, timeout=300)
Expand All @@ -57,7 +55,7 @@ def download_driver(driver_ver, dest_folder):
def unzip_driver_to_target_path(src_file, dest_path):
with zipfile.ZipFile(src_file, 'r') as zip_ref:
zip_ref.extractall(dest_path)
logger.info("Unzip [{}] -> [{}]".format(src_file, dest_path))
logger.info(f"Unzip [{src_file}] -> [{dest_path}]")
Comment on lines -60 to +58
Copy link
Author

@sourcery-ai sourcery-ai bot Oct 26, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Function unzip_driver_to_target_path refactored with the following changes:



def read_driver_mapping_file():
Expand Down
2 changes: 0 additions & 2 deletions crawler/_env.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,5 +8,3 @@
if sys.getdefaultencoding() != 'utf-8':
reload(sys)
sys.setdefaultencoding('utf-8')
else:
pass
Comment on lines -11 to -12
Copy link
Author

@sourcery-ai sourcery-ai bot Oct 26, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lines 11-12 refactored with the following changes:

Loading