Standalone Crawly

[oltarasenko]

Hey,

Occasionally, I have ad-hock projects where I have to extract data from the web. Most of the time, it's all about bringing me a CSV file with data. Ideally, people want the application to run somewhere in their environment, and usually, they don't like to install Elixir and dependencies. They need a tool.

I have an idea of making a Docker version of Crawly using the following approach:

1. Build Dockerfile with Crawly as a released application
2. Use an externally mounted directory with spiders that can be just added [not it's not entirely clear how to compile them if everything else is already a compiled release]
3. Use vm.args to supply a config (as I guess I need to be able to change a set of middlewares, etc)

Any thoughts, or advises?

I have an idea of using crawly as a standalone application. So it's not required to have Elixir installed