URL manipulation

Raed · July 20, 2020, 9:07am

Hi there,

i need help to create a workflow to read or get some restful resources
i have many URL as String the different between them is only in one term
like

www.host.domain/xxx where xxx is placeholder
www.host.domain/yyy where yyy is placeholder
www.host.domain/zzz where zzz is placeholder

the question ist how can i replace the last part of each URL
with the suitable value

xxx = first_value
yyy = second_value
zzz = third_value

where first_value != second_value != third_value

replace the placeholder with the suitable String ?

i look for some Nodes to solve this problem but all of them work with
only one placeholder for all rows in one columen but not multiple

thanks for helping me

Raed

Gavin_Attard · July 20, 2020, 9:31am

Hi,
not sure if i fully understood your intended outcomes, however the string manipulation node allows you to concatenate column values, , so it’s a matter of preparing a table with a domain column and whatever values you have.

If you have an example of the source data and the intended outcome we could help further.

Gavin

Raed · July 20, 2020, 9:47am

Hi Gavin_Attard

thanks alot for your answer
i mean i have many (N) URL the difference is only in one mini String
is there any (one) Node to fill with www.host.domain/* the next Node
shall replace the * sign with the suitable String ?

Node 1 has one row --> www.host.domain/*
Node 2 has 3 rows

xxx
yyy
zzz

through the iteration in the second Node will build the whole URL ?
then with Get Request Node read each URL resource.

Get_Restful_Resource

the problem all Nodes for manipulting a String could replace one and only one sign (*) in many rows
what if i have many different signs need to replace them with different String?

Java Snippet has the following code:

String url = $column0$;
if (url.contains(“asp”)){
return url.replace(“asp”, “aspirin”);
}else if (url.contains(“glu”)){
return url.replace(“glu”, “glucose”);
}else{
return url;
}

Gavin_Attard · July 20, 2020, 10:39am

Ok, i think i get you, you want a kind of ‘Find and Replace’ node, that can iterate through all possible values in the string and replace accordingly.

You can use a series of String Replacer nodes. It’s easier to manage than a java snippet.

You can also try looping around this to iterate around a number of find replace terms.

Raed · July 20, 2020, 10:49am

Hi Gavin_Attard

thanks alot as i know each Replacer Node can replace only one sign thats mean for each
String i need one Replacer Node ? (String Replacer Dictionary may be is the solution)
i mußt have a better look at how to iterate through Strings with Knime Node ?

if you have any Workflow it will be nice in any case thank you so much.

Raed

Gavin_Attard · July 20, 2020, 11:04am

Yes i was thinking one node per string to replace.

The string replace dictionary could also work, I’ve not used it before so not certain on the outcome.

A task like this almost needs a hybrid as you may want to build the ‘dictionary’ in flow rather than write to a file… the read again…

I’d give it a go see what happens.

Raed · July 20, 2020, 11:08am

Hello Gavin_Attrad

thanks alot again i will try some Nodes may be get a solution thank you so much for
your helpful discussion.

Raed

ipazin · July 20, 2020, 1:39pm

Hello @Raed,

welcome to KNIME Community!

Can this be a job for Rule Engine node and LIKE operator?

URL_column LIKE “*asp” => “aspirin”
URL_column LIKE “*glu” => “glucose”
…
TRUE => URL_column

Then use String Manipulation node and join() function to have URL you want.

Br,
Ivan

Gavin_Attard · July 20, 2020, 1:41pm

Hi Raed

I think this should do what you are after.
find-replace.knwf (37.0 KB)

Similar to your idea of using the java snippet, in this case, i used a lookup table. The values are ‘grouped’ into a list,then appended to each row.

The java snippet then loops through each list value replacing as necessary.
It is quite crude, but should do the job

Raed · July 21, 2020, 8:23am

Hello Gavin_Attard

very nice i tried with String Replacer(Dictionary) but it does not work can’t replace part of URL
it replaces the whole URL but not a sign like * or ?

i will download your workflow and test it thanks alot.

Raed · July 21, 2020, 11:53am

Hello @ipazin,

i need to create 3 Nodes the suggestion with Rule Engin was OK it filters the String but i am not sure if the String Manipulation Node would be work with more than one expression because i got can’t complie Error ?

Gavin_Attard · July 21, 2020, 12:11pm

you need a semicolon after each line

Raed · July 21, 2020, 12:36pm

Hi Gavin_Attard

of course i do that after each line but i get an Error ?

Gavin_Attard · July 21, 2020, 12:38pm

have you tried the workflow i sent you yet?

Raed · July 21, 2020, 12:41pm

Hi Gavin_Attard

i get an Error i think because i am using Knime 4.1 may be your workflow for 4.2 ?

Gavin_Attard · July 21, 2020, 12:42pm

Yes correct i am on 4.2

ipazin · July 21, 2020, 1:17pm

Hi @Raed,

This is what I had in mind:
2020_07_21_String_Repalcement.knwf (12.9 KB)

Though still not sure what your input data structure is together with desired output…

In String Manipulation node you can not have multiple expressions.

Br,
Ivan

Raed · July 21, 2020, 1:42pm

Hello @ipazin

if i have the follwing URL

https://pubchem.ncbi.nlm.nih.gov/rest/pug/compound/name/**asp**/cids/TXT
https://pubchem.ncbi.nlm.nih.gov/rest/pug/compound/name/**glu**/cids/TXT

i need to change (Replace)

asp => aspirin
glu => glucose

i think your workflow works only if you want to replace the END of any URL but not the middle of it ?

best regards

ipazin · July 21, 2020, 1:48pm

Hi @Raed,

then you need to modify it a bit
2020_07_21_String_Repalcement.knwf (12.8 KB)

This would be a use case for Cell Replacer with some wildcard/regex support.

Br,
Ivan

Raed · July 21, 2020, 1:54pm

Hello @ipazin

really i am new user of Knime but i need some help to cope with it or learn about the nodes and what each one does