Skip to content
This repository was archived by the owner on May 9, 2018. It is now read-only.

JonnyFb421/wallscrape

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

20 Commits
 
 
 
 
 
 
 
 

Repository files navigation

Wallbase Scraper
v0.4
This script will download wallpapers from the global toplist of Wallbase.cc.  This script is an exercise for me to get a better understanding of Python, threading, and queues.  

To Do
Add URL back to queue if the HTTP request times out so user does not miss any downloads. Now that the worker threads spawn in range of queue.size() if I add it back to the queue it should try to downloading that again without skipping a beat.

Rename class TestyClass.  This class is responsible for the script being a bit slower than it could be.  It's subclass of HTMLParser currently using handle_data and using regex on all that HTML.  The solution to this is only pass the data within the <script> tags instead of ALL the data. 

Add ability to log in

Change Log 


0.4 Fixed issue with threads hanging

About

No description, website, or topics provided.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages