In KNIME, Is there a way to search for non-ascii characters in a csv file

[sgilmour]

In String Manipulation it is listing all my columns as one big column. How do I get them to list as seperate columns.
Thanks
Scott

[ipazin]

Hi @sgilmour,

so if you check your data after File Reader node you’ll see that all your data is in one column. You can either define column separator in File Reader node or use Cell Splitter node after you read your data. Instead of column_name in String Manipulation node you should then use real column name.

Br,
Ivan

[sgilmour]

The Cell Splitter worked. and then I did the column list loop start node

Then I added the node string manipulation and it adding all my columns as 1 again I was going to add the regex expression to each column.

Here is what I have so far for the workflow.

[ipazin]

Hi @sgilmour,

check Split input column name for output column names in Cell Splitter node to have proper column names (if haven’t already). Also you can filter columns you don’t need prior to loop. Then check example workflow I linked in one of my previous replies regarding how to loop over multiple columns and applying same manipulation over each one of them.

Br,
Ivan

[sgilmour]

Hi Ivan,
Is this what you were thinking. I am still working on configuring but wanted to make sure I am on the right track.

[sgilmour]

Hi Ivan,
I modeled my setup after the Looping over all columns and manipulation of each workflow. I am getting stuck when I reach the Math Formula Node. it is giving me an error no such column: Anonymous column when it is added in n the previous node.
Column List Loop Start

Then the Column Rename

Then the Math Formula Node

Then I get the error

Do I need to add something to my workflow to get it to work?

[sgilmour]

sample output of file.
test_output.xls (26 KB)

[ipazin]

Hi @sgilmour,

yes! That is the one I was thinking of. You need to use String Manipulation instead of Math Formula node. Also seems to me you should use Column Filter node to remove all these columns you have in Exclude in Column List Loop Start node.

Br,
Ivan

[sgilmour]

If I add Column Filter it just lists all my columns as one big column and String manipulation is giving me errors.

Here is what I have so far for a workflow.

_Looping_over_all_columns_and_manipulation_of_each 1.knwf (23.0 KB)

[ipazin]

Hi @sgilmour,

you need Column Filter prior to loop start. Remove all columns you don’t need.

Br,
Ivan

[sgilmour]

Doesn’t seem to like my string expression. Plus not sure if I should use the names of my Columns since they are not listed in string manipulation or use the Anonymous_column name.

[ipazin]

Hi @sgilmour,

your expression is fine only has one “)” more than it should.

Br,
Ivan

[sgilmour]

Hi Ivan,
Another issue I noticed with the new column after seeing the CSV file is that now NO LOT NO is NOLOTNO with no spaces in between and I have a Town Acworth and in the new column it is Acorth because it removed the w from the name.

[sgilmour]

Hi Ivan,
When I have more than one item in a column then it is messing up my rows and adding rows.
I changed my regex regexReplace($ANONYMOUS_COLUMN$,"[^u0000-u007F]+/n" ,"" ) by adding /n for newline but this didn’t help. I did this to remove any /n when I import it into my python program.

I will give you an example of what is happening:

So you can see it is messing up my rows when it is has more than one item in a row, and also it is removing the numbers with leading 0’s.
I would like it to keep leading 0’s since that is part of the ID #'s How can I fix this issue,
Thanks again for your help,

[ipazin]

Hi there @sgilmour,

you should definitely check that regex. It is one I found on Internet and haven’t tested it. Especially considering you have more lines in single cell. Leading zeroes are removed in case string column is converted to number column. Maybe that is happening in one of your nodes.

Br,
Ivan

[sgilmour]

Thanks I will keep searching so far no luck

[sgilmour]

Thanks Ivan,
I think I will start a new post for the regex issues

[sgilmour]

The solution above fixed my issue with the Workflow as I was able to run the workflow without errors.
I did have issues with regexReplace but I started a new post to resolve this issue.

[ipazin]

Hi @sgilmour,

glad you made it to some degree

Br,
Ivan