OrientDB upsert for Edges not working

[hermyknime]

Hi All,

i’m trying to upload incrementally some data in OrientDB via the upsert functionality.
It all works fine for Classes (Vertexes) but there seems to be an issue for Edges (as you can’t upload them via the “OrientDB Command Node” same way how you would select a Class.

So I’m using the “OrientDB Execute node” to craft the query

apologies for the ‘deprecated’ nodes usage, i had upgraded the flow from 4.0.0 to 4.5.2

→ successful for a couple of iterations – until there are duplicates

ERROR OrientDB Execute 3:479 Execute failed: Cannot index record #100:140269: found duplicated key ‘OCompositeKey{keys=[#74:12, #65:127976]}’ in index ‘_belongsTo.in_out’ previously assigned to the record #100:138809
DB name=“audienceplatform_combinedsegments_inmem”

when i try to build the query upfront (in the string manipulation node) and run it through the “OrientDB Command node” , it seems that functionality is not implemented…

ERROR OrientDB Command 3:516 Cannot execute future!
ERROR OrientDB Command 3:516 Cannot process commands from datatable!
WARN OrientDB Command 3:516 Node created an empty data table.

→ is this an issue that the OrientDB connect nodes developer (se.redfield) needs to resolve? or might there be a graceful solution?

in order to use the ‘upsert’ we have to have a ‘unique’ index on the Edge as below
http://www.orientdb.org/docs/3.2.x/sql/SQL-Create-Edge.html

Thx

Herman

[Artem]

Hello @hermyknime

As a person who was managing the development of OrientDB extension I can tell, that these nodes are outdated. This way I would assume that many operations are not available or limited.
Due to low demand we do not maintain this extension anymore, unfortunately OrientDB did not become a popular product.

[hermyknime]

@Artem , thx for the info !

[hermyknime]

just to be complete, if others are also looking for this issue…

the SQL statement (via flow variable) in the “OrientDB Command node” does not seems to work either, somehow there is a JSON parser issue when serialising the query to be sent to OrientDB server.

H.

[hermyknime]

one work-around is to

• fetch back the edges already loaded for the scope Vertexes you’d want to load
• dedup them before uploading them to OrientDB

/!/!\ beware of cardinality, or you might bring back your entire database into Knime

• e.g. here I fetch edges for accounts in scope, rather than fetching edges for segments in scope. As a segment can contain millions of accounts.

MATCH
{class: Account, as: account , where: (ACCT_SK_ID in [ $${SACCT_SK_ID (Concatenate*)}$$ ] ) } -_belongsTo-> {class: Segment, as:segment }
RETURN account.@rid as ‘@rid_ACCT_SK_ID’, segment.@rid as ‘@rid_SOURCE_SEGMENT_ID’
;

→ beware: back-ticks