Reading PDF files

[LakshmiK]

Hi,

First of all, this is a great community!
I am looking for help to read a .pdf document (such as Annual reports). I tried Pdf parser and Tika parser and then did a ‘Bag of Words Creator’ to get the total list of words (using which I am required do some further analysis) . But both instances gave different number of total words, which is different from the actual figure. I tried converting the pdf to Word and then used the Word parser. That gave me another different number. I am not so well versed with all this, am I missing something here? Please
help.

Thank you!

[aworker]

Hi @LakshmiK

If your PDF documents are not confidential, could you please upload here your workflow (does not need to be run) and your documents inside the workflow ? Usually, the best place to store the data is inside the workflow in a directory located at its first level called “data”.

Best
Ael

[mlauber71]

@LakshmiK maybe you could take a look at these examples reading words and data from pdf with the help of R packages:

Unstructured text mining from PDF

@janszky_nav I would suggest you toy around with the workflow read the comments and the links. I was not able to extract the PDF from the word file but used the file you posted earlier (Text mining from PDF documents and results places - #6 by janszky_nav). I placed your PDF file in the /data/ folder of the node and searched for it in order to create the paths. If you have more than one PDF they would all be processed. The paths are also fed to R so no need to enter them manually. You also do n…

[LakshmiK]

Thank you @aworker and @mlauber71 . I will do that. It will take me a bit of time to comprehend this.

Meanwhile, I think I may have understood something wrongly and am working on it. But thank you again so much! I am afraid I will most probably need further help as I proceed with the data and will need to reach out again.

Regards,
Lakshmi

[julian.bunzel]

Hi @LakshmiK,

The PDF Parser and the Tika Parser both use the same library so I would expect the same output as well here. Are you using different tokenizers in the String to Document node? Additionally, you do not need to use the Document Data Extractor and then use the String to Document node again for the output of the PDF Parser as it already converts the data to the Document data type.

To see what is going wrong and which words are different, you can use a Term to String node to turn the Terms into a String column. Afterward, do a full outer join with a Joiner node. This will help to see which words exist in each Bag Of Words and which are missing.

[LakshmiK]

Thank you @julian.bunzel . Sorry for the delay in replying.
I have avoided Document Data Extractor now. I think I understand a bit better now and my workflow seems to be giving much better results overall. Thank you.

I did a full outer join with the terms from the 2 parsers. There are some words which are extra/ missing in both cases but not too many. I am reading an annual report and these ‘extra’ words are some numbers in both cases and a few terms which (are a combination of number and alphabets and) do not make much sense.

Thank you so much again!