Feature Request: HtmlParser drop Result column after parsing

Dear Palladian maintainers,

I really like your extensions and use them a lot to scrap data off websites which works great BTW!
I have a feature request: It would be nice if the HtmlParser node would drop the Result column after parsing.

Recently I have come across a weird behavior when a node took up 800mb of disk space (and it took ages to save the workflow). The reason being that I used a XPath node on a parsed html and the XPath produced 2000 rows each with a result column of ~ 0.5kb (=800mb). I didn’t know that the result was this big.

Such an option (enabled by default) would have prevented this from happening, and it also would be quite handy because then I would not have to drop in column filters everywhere (I am a lazy user :D).

Cheers,
Johannes

1 Like

Hi Johannes,

thanks for the feedback – I totally agree, this definitely makes sense. I put it on our todo-list!

Best,
Philipp

2 Likes