ipv6 conversion

Hello guys,

I would like to ask you one very important question for me.

At now days I have to deal with ipv6 addresses. Transform them and enrich by some additional very useful information such country code, country, coordinates etc.

To be able do this action I need to convert ipv6 into decimal format. Unfortunately this format consists of 38 digits (compare to ipv4 which has only 15 digits).

The knime workflow is attached and contains just two working nodes:

table creator: contains all possible compressed ipv6 addresses
java snippet - IPv6 Zero Decompress, decompression of compressed ipv6 to full ipv6 format.

And now problem comes.

In the second java snippet called Convert IPv6 to Decimal format. But due to so long number (38 digits) is not possible to convert it successfully directly to longinteger or to double or some another numeric format.

Can someone help me how to solve this problem?

Once I have decimal number I am able to use node Binner (dictionary) and make range look up and returns country to particular ipv6 address.
Example of country list bellow:

Start Number - End Number - Country code - Country name

“42540528726795050063891204319802818560”,“42540528727168608142152124734787026943”,“JP”,“Japan”

For ipv4 it works perfectly. Can someone help me please? I think this workflow could be very useful for many people.

Thank you

ipv6-test.knwf (11.3 KB)

Hi @sm0lda

The problem is it 's impossible to have more than 15-16 digits of precision for double values in Java. Long values can hold up to 18 digits. That means only your first 16 (18 in case of Long datatype) most significant digits will be reserved. In the case of Double, the others are replaced with zeros, because that is the closest possible value. In the case of Long values with more digits than allowed turn in to something “unexpected”. If you must represent all 38 digits coming from an IPv6 value you need at least 2 Long / 3 Double values.

Hi @temesgen-dadi,

ok that’s something I didn’t want to hear, but honestly said I expected it. But how can i solve it? There must be some way. Let’s imagine I use 2 Long, but how can I make binning and range look up to get country information if I have to split long numbers?

Theoretically HEX format as one number could be fine, but how to convert uncompressed ipv6 to HEX format? Any idea? In the best case not to use java. Just with knime nodes.

Thanks a lot!

Hi @sm0lda

Once you decompress the ipv6 values and make sure they have equal length and converted to uppercase, you can use the Binner (Dictionary) node with string values as well. The comparison/lookup is lexicographical. Here is your workflow modified with a toy lookup table.

ipv6-test-with-lookup.knwf (14.8 KB)

Best,
Temesgen

1 Like

Hi @temesgen-dadi,

thanks for positive info about Binner (Dictionary) this node works even with string. I have to try.

I have an idea to split uncompressed ipv6 address into two parts and process them separately and at the end use number to string for both of them + string operation to merge them into one long string and use Binner (Dictionary). it seems like problem is solved.

But now comes the challenge. How to convert those two parts of ipv6 to 2x 64bit number?

i can not find any formula how to do that.

Any idea?

Thank you

Hi @sm0lda

regarding convertinv hex to decimal, have a look at this thread from the forum.

Do you necessary need to split, convert to decimal, combine back? Does your lookup table come only in decimal format? I am asking because the Binner (Dictionary) node can work directly on the expanded hexadecimal representation without the need to split it.

Best,
Temesgen

Hi @temesgen-dadi,

I have everything figured out, but now I am facing with some troubles and it is the last step - loop connected with header variable.

image

I have 32 columns named by indexes of number in uncompressed ipv6 HEX format and I need to grab header as variable and use this variable in loop as number in math formula. This is for HEX to DEC conversion.

Something like this:

Example of formula should be
iteration number 1 - 1016^$$column header 31$$
iteration number 2 - 3
16^$$column header 30$$
iteration number 3 - 1516^$$column header 29$$
iteration number 4 - 5
16^$$column header 28$$
.
.
iteration number 32 0*16^$$column header 0$$

It should not so difficult build this loop.

Please give me some advice.

Thank you

Hello @sm0lda,

just checking out if you have managed to figure out how to use column name in Math Formula node? If haven’t you can use flow variable currentColumnName created by Column List Loop Start. This flow variable is of type String so won’t be available in Math Formula node. However you can use Variable Expressions node and toInt() function to create new flow variable of type integer.

Br,
Ivan

Hello Ivan,

nice to hear you. I am at the end. The last step is impossible from my POV finish.

Anyway I struggle with 128bit number at only the first half is calculated. The rest is filled by zeros…

Please check workflow.

ipv6.knwf (35.7 KB)

Any idea please?

Thank you

Jiri

Hello @sm0lda,

here is what I had on my mind although not sure it is what you are looking for…
ipv6_ipazin.knwf (43.6 KB)

Br,
Ivan

wow it is nice, but you have the same result than me. All columns are incomlete. Only half of numbers are there due to 128bit long number.

Do you have any idea how to do it?

Thank you

Jiri

Hello @sm0lda,

Thought this splitting is covering that :sweat_smile:

Br,
Ivan

it covers just conversion of each index.

But

  1. it has calculated just 64bit of 128bit and the rest filled with zeros
  2. I need to join them together and due to the fact I have just half of 128bit length I have no idea how to do that

Jiri

Hello @sm0lda,

are you saying Math Formula node calculated something wrong or? :confused:

Br,
Ivan

Hello Ivan,

please check it out.

Complete number is:
338288524927261089654172166563218180926

Incomplete number is:
338288524927261100000000000000000000000

due to the fact the number its self is too long. This is what I am talking about. Last 24digits are cut, filled by zeros and rounded. I think this issue is not possible to solve.

ipv6_almost_done.knwf (42.4 KB)

Thank you

Jiri

Hello @sm0lda,

seems so. Maybe Python or R integration can help in this case :thinking:

Br,
Ivan

Hello @ipazin,

yes and this is the problem I have mentioned many times.

I am not able to use Python or R, so it seems like this problem is impossible to solve just purely with KNIME… :frowning:

Jiri