Components for Hashing Files and Strings released on KNIME hub

Dear KNIME community,

I am happy to announce that I shared my “Hash Files” and “Hash Strings” components on KNIME hub. These can be used to calculated cryptographic hash values (i.e. unique fingerprints transforming arbitrary length inputs to pseudo-random fixed length outputs deterministically). So far, this was only possible via String Manipulation (but for the MD5 hash function only) or some inofficial extensions.

The standard hash functions like MD5, SHA-1, SHA-2 family and SHA-3 family are supported.
You also have the choice of output in either hex or base64 format.

I developed these components since I ran into multiple use cases that required hashing. Git, blockchain, many file archiving solutions, etc. are based on hashes, so maybe this can help unlock your creativity as well.

Not yet supported (due to the limitations of Java Snippets inside these components) are multi-threading the compute-intense hashing task and supporting the new File System logic of KNIME (so far, ‘Hash Files’ only supports local files).

Best regards

Arne

7 Likes

Hi @arbe , you can check the Palladian Extension which offers several hash functions (among other things):

Here’s a zoom on their Hashcalculator:

You can just use this Extension :slight_smile:

5 Likes

Hi @bruno29a,

I am aware of this option. But working in a corporate environment, the Palladian Extension residing in its own update site (not official on the KNIME update site anymore) is unfortunately often not feasible.
The components on KNIME hub require base KNIME only.

Also, the Palladian HashCalculator requires files to be read as binary data and then to be hashed by this node. This is very memory expensive and not my preferred file hashing strategy, especially for larger files.

Don’t get me wrong: Great work there on the Palladian extension. But my components still provide a value add, that’s why I decided to share them. :slight_smile:

Try it out yourself:
Hash Files component on KNIME Hub
Hash Strings component on KNIME Hub

1 Like

hi, @arbe , Thanks for this component.

Btw, add the Path type support(current only support string type) to the hash files component could be more complete.

Thanks again!

Hey Arne,
thanks a lot for this component, came in very handy for my use case. Well fleshed out, many options and really fast.

1 Like