Description

Fetches HTML content from one or more URLs and stores it in a cache (Redis), acting as a basic search engine indexer.

Running the app in docker

docker compose -f 'docker-compose.yml' up -d --build

Try the API :

POST :

curl --request POST \
  --url http://localhost:3000/ \
  --header 'content-type: application/json' \
  --data '{
  "urls": [
    "https://www.qwant.com/"
  ]
}'

GET :

curl --request GET \
  --url 'http://localhost:3000/?url=https://www.qwant.com/'

Running the app in local

Installation

$ npm install

Test

# unit tests
$ npm run test

# e2e tests
$ npm run test:e2e