Add examples for new quotes.toscrape.com endpoints (fixes #15)

[rjanain]

This PR adds spiders for 6 new endpoints available on quotes.toscrape.com, providing comprehensive coverage of modern web scraping techniques. Currently, quotesbot only covers 2 out of 8 endpoints on the sandbox - this update brings complete endpoint coverage.

Closes #15

The quotes.toscrape.com sandbox has evolved significantly with new endpoints designed to teach modern web scraping challenges. While quotesbot has remained an excellent starting point with CSS and XPath examples, it doesn't demonstrate techniques for JavaScript rendering, APIs, authentication, and other scenarios that students encounter in real-world scraping projects.

Changes

1. New Spiders for Modern Endpoints

JavaScript & API Handling:

• toscrape-js.py → /js/ endpoint

  • Extracts data from JavaScript-rendered content
  • Parses JSON embedded in <script> tags as var data = [...]
  • Handles pagination with JS-generated content
  • Demonstrates reverse-engineering client-side rendering

• toscrape-scroll.py → /api/quotes?page=N endpoint

  • Consumes REST API directly instead of HTML scraping
  • Parses JSON responses with proper pagination handling
  • Shows when to use API endpoints over HTML parsing
  • Demonstrates API pagination with has_next and page fields

Authentication & Forms:

• toscrape-login.py → /login endpoint

  • Demonstrates complete authentication flow
  • Uses FormRequest.from_response() for automatic CSRF handling
  • Validates successful login before scraping
  • Maintains session across paginated requests

• toscrape-viewstate.py → /search.aspx endpoint

  • Handles ASP.NET ViewState forms
  • Extracts and submits __VIEWSTATE hidden fields
  • Demonstrates stateful form submissions
  • Teaches techniques for legacy enterprise applications

Complex Layouts:

• toscrape-table.py → /tableful/ endpoint

  • Parses quotes in HTML table structure
  • Demonstrates table scraping patterns
  • Shows iteration through rows and cells

• toscrape-random.py → /random endpoint

  • Scrapes endpoints with dynamic/random content
  • Simple example for non-deterministic sources

2. Updated Existing Spiders

• toscrape-css.py and toscrape-xpath.py:
  • Now use QuotesbotItem instead of plain dicts for consistency
  • Replaced deprecated .extract_first() and .extract() with .get() and .getall()
  • Maintains backward compatibility in functionality

3. Enhanced Data Model

• items.py: Added explicit field definitions for text, author, and tags
  • Makes the item structure clear and educational
  • Previously just had pass with commented placeholder

4. Comprehensive Documentation

• README.md:
  • Added "About This Project" section explaining endpoint coverage
  • Maps each spider to its specific endpoint URL
  • Organizes spiders by technique category (HTML, JS/API, Auth, Layouts)
  • Provides learning path progression from basic to advanced
  • Includes direct endpoint URLs for hands-on exploration
  • Added installation instructions and example commands