Getting error details from API through POST Request

[kowisoft]

Dear KNIMErs,

I am currently running a test connecting my local KNIME Analytics Platform to an internal API service. It’s basically a reference check if a certain element exists or not.

Now my problem is this:
When the element exists, I get back a status 200 plus details.

But when the element doesn’t exist, the API provides a status 400 and error details. I need to extract those details, but - looking at the KNIME Post Request node - I cannot see any options to receive error details.

All I can see the “standard” error cause, which for an error 400 would simply give me “bad request”. The problem is that this is not specific enough. Because I cannot say, if this is really because of a bad request (maybe I forgot some forward slashes in the POST Request node configuration) OR because the reference check failed.

The error details are structured like this.

For e. g. missing a mandatory element in my POST request (a REAL bad request):

{
  "errors": [
    {
      "detail": "\"\" is not a valid choice.",
      "status": "400",
      "source": {
        "pointer": "/data/attributes/referenceType"
      },
      "code": "invalid_choice"
    }
  ]
}

versus a failed reference check (the element does not exist):

{
    "errors": [
        {
            "detail": "line-item-id (12345) DOES NOT EXIST IN ABC-TABLE",
            "status": "400",
            "source": {
                "pointer": "/data/attributes/1"
            },
            "code": "sys_error_103"
        }
    ]
}

The latter, I could extract and turn into a table and then act accordingly, whereas in the first I know, I have misconfigured my node.

Any suggestions on how to get the error details from the response outputted in KNIME?

Thank you in advance!

[thor_landstrom]

Hey @kowisoft,
Just to make sure I understand, you are trying to fetch the details from the json output from the bad request? I think you might be looking for this option:

Let me know if this is what you were looking for,
TL

[kowisoft]

Hi @thor_landstrom ,

yes, exactly.

For context: I am sending internal order details to a reference API to check if these order details are correct. They follow a certain pattern, however, we have 100s or 1000s of them and they often come from manual entries. This means in return, they might be error prone and for that we have an API that checks the pattern and returns 200 - yes, this is a valid reference or e. g. 400 - no, this is NOT a valid reference.

So here are a few more details about the responses my colleagues (who run the API service) told me:

in the case of a 400 error, the detail/source/code attributes in the response should give more context and help to figure out if the error is due to a misconfigured POST request or not.

For instance, this is the response that I get if I a misconfigured POST request is this:

{
  "errors": [
    {
      "detail": "\"\" is not a valid choice.",
      "status": "400",
      "source": {
        "pointer": "/data/attributes/referenceType"
      },
      "code": "invalid_choice"
    }
  ]
}

This basically indicates that I forgot one important argument for the API called “reference type” (this is specific to this very API)

And this would be the response I would get, if I send a “correct” POST request but try validate a reference that is not there

{
    "errors": [
        {
            "detail": "ORG-ID (ABCD1234) DOES NOT EXIST IN ABC-TABLE",
            "status": "400",
            "source": {
                "pointer": "/data/attributes/1"
            },
            "code": "orc_error_103"
        }
    ]
}

So the idea is to basically separate (and afterwards split) those 400 results into a list which we can return to the people telling them: hey, there is something wrong with the entry, whereas the 400 errors e. g. with invalid choice are a technical thing we could clarify ourselves in the workflow.

Coming back to your response: I have checked the “Output additional column with error cause” but this just returns the standard message of “Bad request” and not the expected details as per the examples above.

Hope that makes sense.

[thor_landstrom]

So when you checkmark that box you are not getting the JSON body back from the REST request? I was thinking something like this originally should be what you are looking for:

Where you just use JSON Path, and extract the message, or in your case “detail” from the json response

[kowisoft]

No, I do get an empty body back.

[thor_landstrom]

@kowisoft,

Have you tried curling the endpoint (or alternatively using postman or similar)? I would try this to see if you are getting anything back from the api. It could be an issue on the API side. If not I can look into why the node may not be returning the body.

TL

[kowisoft]

I tried it with Postman and it actually returns the expected error text:

[thor_landstrom]

It will be a bit hard for me to narrow it down as I cannot access the endpoint to play around with it, but I can make a simple REST endpoint actually and return a response similar to the body you expect. If I run into the knime node not getting the body, I believe a ticket will need to be made for further investigation.

Maybe the node doesn’t like that its inside a collection? As it seems non standard for the message to be placed in it, but I will let you know what I get.

TL

[kowisoft]

before you do that, let’s maybe see if we can find something on fakestoreapi or something to see.

If we create non-standard-compliant APIs it should most definitely not be your problem

[thor_landstrom]

I already created one, I get a response like so using postman:

I tried to make it as similar to yours as possible, when I try to ping it via KNIME:

I do get the response in the error cause. The body is empty though; I do not think its the json then since I am able to get it.

[kowisoft]

I think we can forget this now, because it seems the config of our internal API is simply not very good. I do also get an error cause, but only “Bad Request” nothing else.

Tbh, I am bit frustrated by the way the IT in our company has set this up and I don’t want to find some super strange workarounds because of their inability to develop proper APIs.

[thor_landstrom]

I do find it weird how you can see the body in the postman but not in the knime node. But, if you do find anything repeatable where the POST request can’t correctly parse the body please send it over. Maybe the API is doing something else that I don’t do on my end.

TL