I've been wanting to run my own search engine sorta thingy that indexes websites I feed it. I sometimes find little nooks of the net that post resources I may need in the future. Like my own mini-Google that indexes a list of sites.

How can I go about creating this? Are their off-the-shelf solutions, will I need to say combine scrapy with elastic search? The links in this thread look promising.