PDF and Tika Parser

[Nazareno]

Hi,
Scientific pdf files are usually composed of 2 or 3 columns in one page. When I use the PDF or Tika parser and I check the text content output afterwards, these columns are combined (attached picture).
Do you know whether would be possible to avoid such a thing and read the columns separately?
Thank you in advance!
Cheers,
Nazareno.

[julian.bunzel]

Hey @Nazareno,

unfortunately I don’t see any (easy) possibility to work around this issue right now.
I will create a ticket in order to fix this.

Thank you for reporting.
Cheers,
Julian

[Nazareno]

Hi Julian,
Thank you for the answer. I hope you can solve it soon.
Best,
Nazareno.

[cscheeser]

hi all - any update on this? maybe a number of characters by row? or some kind of find and replace?

[ScottF]

Hi @cscheeser -

I checked the ticket (AP-14318) and there’s nothing new to report unfortunately. But I will add a +1 from you on it.

[cscheeser]

Hi Scott thx for the reply. I can envision some kind of grid system (maybe in 0.1" increments) where text plucked from the section of x-y space would be imported as contiguous. Other text ignored.

Related if you have any clever regex or similar hacks, would love to learn more and see what might possible with existing nodes…

[ScottF]

Our expert on text extraction from PDFs is @victor_palacios - I tag him here in case he has some clever tricks to share in this case.

[victor_palacios]

Hello @cscheeser ,

This is an image segmentation problem and not strictly a PDF-parsing problem. You can see here my response to something very similar:

Associate text to images from PDF

This seems to be an image segmentation problem. Here is an image processing webinar which includes segmentation. Around 0:40 they begin constructing this: Solution – KNIME Community Hub The key here may be the splitter node depending on your data. Once you’ve segmented the images, then you can apply OCR and have the rows align with the image. I don’t recommend you do it manually, but hopefully the videos will teach you a more programmatic way to do so. If not, you may require a neural networ…

Being able to do this kind of segmentation is actually state of the art, so it would require the advanced techniques shown in the link above.

[cscheeser]

Thank you Victor - i’ve seen another video of yours, good stuff!

I’ve had good luck with opportunistic parsing downstream of the Tika Parser. One area I can’t resolve - wrapping text in columns that falls to the next row, see pic:

upon parsing, i see the below. row 1 and row 2 both fine. row 3 though is too anonymous. Any ideas welcome.

DC WHSE NO. DATE NO. NO. NO. DESCRIPTION DESCRIPTION QTY PRICE AMOUNT
01/10/2023 0 0 FOF Oral Care FOF Oral Care 1.00 16,154.00000 16,154.00
Incremental Incremental SKU funding

[victor_palacios]

Hello @cscheeser , could you give me a little more detail? I’m not sure what you mean by “anonymous”.

[cscheeser]

Hi - per the pic, text wrapping occurs in Column 8 and Column 9. However the post-parsing data suggests row 2 of text wrapping occurs in Column 1. No spaces, no tabs, no delimiters suggest the wrong column.

How to get the column 8’s “Incremental” to line up with column 8? “Incremental Sku Funding” in column 9?

[cscheeser]

another pic of the data post parser

again, row1 and 2 and manageable, lots of delimiters & patterns baked in. but row three isn’t able to attributed to column 8 or column 9. relationship seems lost…

[victor_palacios]

@cscheeser

Now I understand: when text appears within in a box on 2 different lines like “Incremental” it actually gets assigned a new row and that new row’s column is incorrect. This seems to be an issue with parsing a table which is notoriously difficult. See the many discussions I’ve had with people about reading from tables:

*Almost* Successful: Amazon Textract for Automated Invoice Processing

I managed to get my first Amazon Textract workflow running for Automated Invoice Processing! Man, installing the python integration took me back to installing and running ubuntu packages. I have some issues on the OCR side of things as well as general questions on the endpoints of the workflow. Here’s the start point (Automatic Invoice Processing using Amazon Textract through Boto3 – KNIME Community Hub) Here’s a snapshot of source invoice [image] Current issues Date reads as Feb10/2: …

Reading Table from PNG with Tess4J

@wisemanleo , So there are 2 problems here: Can we read the data in? Even if the data is read in, can it be OCR’ed correctly to transform it into a table? The short answer to 1) is yes, we can read the png. But 2) seems to be no, we cannot OCR this data to get a reasonable output with KNIME. The long answers: –1– For the image reader throwing errors, I also had the Image Reader behave unexpectedly (sometimes throwing errors, sometimes not). Instead Iet’s use the Image Reader (Table) node.…

Analyse PDFs

It may not be directly solve your task but may be helpful to identify surrounding terms

TLDR; Not even state of the art models can read from tables with 100% accuracy without specific training, so this is one area where manual effort, clever strategies, or advanced models need to be used.

[cscheeser]

Thx Victor - appreciat this reply. One idea: Adobe pro has a ‘batch’ exporter. If you batch export to excel, formatting seems retained. i’m playing w/ this now (of course you have to have a license). But maybe the solution is shared between adobe pro and knime?