Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion python-examples/auth-with-secret-otp/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ Reference for saving project [here](https://docs.intunedhq.com/docs/02-features/

## Auth Sessions

This project uses Intuned Auth Sessions. To learn more, check out the [Authenticated Browser Automations: Conceptual Guide](https://docs.intunedhq.com/docs/getting-started/conceptual-guides/authenticated-browser-automations-conceptual-guide).
This project uses Intuned Auth Sessions. To learn more, check out the [AuthSessions](https://docs.intunedhq.com/docs/02-features/auth-sessions).

### Create a new auth session
```bash
Expand Down
2 changes: 1 addition & 1 deletion python-examples/e-commerce-auth-scrapingcourse/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,7 @@ uv run intuned deploy

## Auth Sessions

This project uses Intuned Auth Sessions to maintain authenticated access to the dashboard. To learn more, check out the [Authenticated Browser Automations: Conceptual Guide](https://docs.intunedhq.com/docs/getting-started/conceptual-guides/authenticated-browser-automations-conceptual-guide).
This project uses Intuned Auth Sessions to maintain authenticated access to the dashboard. To learn more, check out the [AuthSessions](https://docs.intunedhq.com/docs/02-features/auth-sessions).

### Create a new auth session
```bash
Expand Down
3 changes: 2 additions & 1 deletion python-examples/e-commerece-category/Intuned.jsonc
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,8 @@
"metadata": {
"template": {
"name": "e-commerece-category",
"description": "E-commerce category and product scraper"
"description": "E-commerce category and product scraper",
"tags": ["web-scraping", "e-commerce", "categories", "extend-payload"]
},
"defaultRunPlaygroundInput": {
"apiName": "ecommerece-category",
Expand Down
152 changes: 52 additions & 100 deletions python-examples/e-commerece-category/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,11 +2,17 @@

E-commerce store scraper template for sites with category-based navigation. Scrapes categories, product listings, and detailed product information.

## Run on Intuned
## Key Features

- **Category-Based Navigation**: Automatically discovers and scrapes all product categories from store menus
- **Three-Step Workflow**: Categories → Product Lists → Product Details using `extend_payload` chaining
- **Dynamic API Chaining**: Each API automatically triggers the next step in the scraping workflow
- **Comprehensive Product Data**: Extracts titles, prices, sizes, descriptions, shipping, and returns information

Open this project in Intuned by clicking the button below.
<!-- IDE-IGNORE-START -->
## Run on Intuned

<a href="https://app.intuned.io?repo=https://github.com/Intuned/cookbook/tree/main/python-examples/e-commerece-category" target="_blank" rel="noreferrer"><img src="https://cdn1.intuned.io/button.svg" alt="Run on Intuned"></a>
[![Run on Intuned](https://cdn1.intuned.io/button.svg)](https://app.intuned.io?repo=https://github.com/Intuned/cookbook/tree/main/python-examples/e-commerece-category)

## Getting Started

Expand All @@ -26,121 +32,67 @@ After installing dependencies, `intuned` command should be available in your env

### Run an API
```bash
# Scrape categories
uv run intuned run api ecommerece-category .parameters/api/ecommerece-category/default.json

# Scrape product list
uv run intuned run api ecommerece-list .parameters/api/ecommerece-list/default.json

# Get product details
uv run intuned run api ecommerece-details .parameters/api/ecommerece-details/default.json
```

### Deploy project
### Save project
```bash
uv run intuned deploy
uv run intuned run save
```

Reference for saving project [here](https://docs.intunedhq.com/docs/02-features/local-development-cli#use-runtime-sdk-and-browser-sdk-helpers)


### Deploy project
```bash
uv run intuned deploy
```

### `intuned-browser`: Intuned Browser SDK

This project uses Intuned browser SDK. For more information, check out the [Intuned Browser SDK documentation](https://docs.intunedhq.com/automation-sdks/overview).

<!-- IDE-IGNORE-END -->




## Project Structure
The project structure is as follows:
```
├── api/ # Your API endpoints
│ ├── ecommerece-category.py # API to scrape category links from the main menu
│ ├── ecommerece-list.py # API to list all products from a category page
│ └── ecommerece-details.py # API to fetch detailed product information
├── utils/
│ └── types_and_schemas.py # Pydantic models for type validation
├── Intuned.jsonc # Intuned project configuration file
└── pyproject.toml # Python project dependencies
/
├── .parameters/ # Test parameters for APIs
│ └── api/
│ ├── ecommerece-category/
│ │ └── default.json
│ ├── ecommerece-list/
│ │ └── default.json
│ └── ecommerece-details/
│ └── default.json
├── api/ # API endpoints
│ ├── ecommerece-category.py # Scrape category links from menu
│ ├── ecommerece-list.py # List products from category page
│ └── ecommerece-details.py # Extract detailed product info
├── utils/ # Utility modules
│ └── types_and_schemas.py # Type definitions and Pydantic models
├── Intuned.jsonc # Intuned project configuration
└── pyproject.toml # Python project dependencies
```

## How It Works

1. **ecommerece-category** - Entry point that scrapes category links from the store's main menu
2. **ecommerece-list** - Scrapes product listings from each category
3. **ecommerece-details** - Fetches detailed product info (title, price, sizes, description, shipping/returns)

Each API uses `extend_payload` to chain to the next step in the scraping flow.


## `Intuned.jsonc` Reference
```jsonc
{
// Your Intuned workspace ID.
// Optional - If not provided here, it must be supplied via the `--workspace-id` flag during deployment.
"workspaceId": "your_workspace_id",

// The name of your Intuned project.
// Optional - If not provided here, it must be supplied via the command line when deploying.
"projectName": "your_project_name",

// Replication settings
"replication": {
// The maximum number of concurrent executions allowed via Intuned API. This does not affect jobs.
// A number of machines equal to this will be allocated to handle API requests.
// Not applicable if api access is disabled.
"maxConcurrentRequests": 1,

// The machine size to use for this project. This is applicable for both API requests and jobs.
// "standard": Standard machine size (6 shared vCPUs, 2GB RAM)
// "large": Large machine size (8 shared vCPUs, 4GB RAM)
// "xlarge": Extra large machine size (1 performance vCPU, 8GB RAM)
"size": "standard"
}

// Auth session settings
"authSessions": {
// Whether auth sessions are enabled for this project.
// If enabled, "auth-sessions/check.ts" API must be implemented to validate the auth session.
"enabled": true,

// Whether to save Playwright traces for auth session runs.
"saveTraces": false,

// The type of auth session to use.
// "API" type requires implementing "auth-sessions/create.ts" API to create/recreate the auth session programmatically.
// "MANUAL" type uses a recorder to manually create the auth session.
"type": "API",


// Recorder start URL for the recorder to navigate to when creating the auth session.
// Required if "type" is "MANUAL". Not used if "type" is "API".
"startUrl": "https://example.com/login",

// Recorder finish URL for the recorder. Once this URL is reached, the recorder stops and saves the auth session.
// Required if "type" is "MANUAL". Not used if "type" is "API".
"finishUrl": "https://example.com/dashboard",

// Recorder browser mode
// "fullscreen": Launches the browser in fullscreen mode.
// "kiosk": Launches the browser in kiosk mode (no address bar, no navigation controls).
// Only applicable for "MANUAL" type.
"browserMode": "fullscreen"
}

// API access settings
"apiAccess": {
// Whether to enable consumption through Intuned API. If this is false, the project can only be consumed through jobs.
// This is required for projects that use auth sessions.
"enabled": true
},

// Whether to run the deployed API in a headful browser. Running in headful can help with some anti-bot detections. However, it requires more resources and may work slower or crash if the machine size is "standard".
"headful": false,

// The region where your Intuned project is hosted.
// For a list of available regions, contact support or refer to the documentation.
// Optional - Default: "us"
"region": "us"
}
```

## APIs

| API | Description |
|-----|-------------|
| `ecommerece-category` | Entry point that scrapes all category links from the store's main navigation menu. Automatically triggers `ecommerece-list` for each category using `extend_payload` |
| `ecommerece-list` | Scrapes product listings from a specific category page. Automatically triggers `ecommerece-details` for each product using `extend_payload` |
| `ecommerece-details` | Extracts comprehensive product information including title, price, available sizes, description, shipping details, and returns policy |


## Learn More

- [Intuned Documentation](https://docs.intunedhq.com)
- [Intuned Browser SDK](https://docs.intunedhq.com/automation-sdks/overview)
- [Web Scraping Recipe](https://docs.intunedhq.com/docs/01-learn/recipes/)
- [extend_payload Helper](https://docs.intunedhq.com/automation-sdks/intuned-sdk/python/helpers/functions/extend_payload)

7 changes: 4 additions & 3 deletions python-examples/ehr-integration-python/Intuned.jsonc
Original file line number Diff line number Diff line change
Expand Up @@ -20,11 +20,12 @@
"metadata": {
"template": {
"name": "ehr-integration-python",
"description": "Electronic Health Records integration example with authentication"
"description": "Electronic Health Records integration example with authentication",
"tags": ["ehr", "healthcare", "auth-sessions", "data-extraction"]
},
"testAuthSessionInput": {
"username": "demo@email.com",
"password": "password"
"username": "Admin",
"password": "admin123"
}
}
}
Loading