SAS Web Crawler - Piping output to custom storage?

[GrokAndRoll]

Hi,

I'm new to Information Retrieval Studio and SAS Crawler, but I was wondering if it was possible to pipe the output of the crawler to a storage place of my choosing - for example, and elasticsearch index?

Cheers,

Seb

[GrokAndRoll · Accepted solution]

Ok I've since found that custom postprocessors can be written in Python and executed as steps in the Pipeline Server. Therefore you could write something in Python which took the document and stored it in, say elasticsearch.

Thanks,

Seb